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ABSTRACT 

The quality of mathematics and science education in 
the United States and the goal of becoming first in the world in 
those two disciplines are topics of concern. This study attempts to 
describe the current condition of mathematics and science education 
and to evaluate the adequacy of e::isting national data to provide 
such a description. The report is presented in six sections. An 
introduction defines curriculum, enumerates the research questions, 
and describes the organization of the report. Sections 2-5 present 
the current status, the availability and reliability of data, and the 
evaluation of the strengths and weaknesses of alternative indicators 
for the following areas: graduation requirements; course 
availability; course completion; and course content. Section 6 
presents conclusions and recommendations of the study. The study 
concluded that the existing data sources were inadequate to construct 
a system of curriculum indicators. The indicator patchwork pieced 
together did not cover all aspects of curriculum, was temporally 
uneven, relied on sources of uneven quality, contained measures that 
were not always congruent, and in sum, was inadequate for effective 
policymaking. The following recommendations were made to fill in data 
gaps and build a comprehensive curriculum indicator system: (1) 
ensure the continued and extended availability of graduation 
standards, course availability, and course completion data; (2) 
develop, validate, and incorporate measures of course content into 
ongoing data-collection efforts; (3) support research to validate and 
extend the information provided by ongoing data sources; and (4) 
develop sources for information to describe the mathematics and 
science curriculum at the elementary and middle school levels. 
Supplemental sections provide a summary, lists of tables and figures, 
principal data sources for buildinr the curriculum indicator 
patchwork, NAEP (Nat ional^ssessment of Educational Progress) 
variables with high rates of missing values in 1985-1986, additional 
tables and figures, and a list of 59 references. (MDH) 
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PREFACE 

How good are mathematics and science education in the United States? How 
adequately do available data describe the quality of mathematics and science education? The 
current widespread concern about weaknesses in the performance of American students and 
the adoption of a national goal of becoming first in the world in mathematics and science 
achievement highlight the importance of answering the first question. Recent efforts to build 
a system of indicators capable of monitoring progress toward this goal underscore the 
importance of the second question. 

These two questions motivated the current study, funded by the National Science 
Foundation, which attempts to describe the conditions of mathematics and science education 
and to evaluate the adequacy of existing national data to provide such a description. This 
work builds upon earlier efforts by HAND to develop models of indicator systems in 
mathematics and science (Shavelson et al., 1987), to examine access to mathematics and 
science curriculum (Oakes et al, 1990), and to develop method?^, of extending existing 
curriculum descriptions (McDonnell et al., 1990). This Note explores the mathematics and 
science curriculum at the secondary level; a companion Note being prepared by Koretz 
examines student achievement in mathematics and science. A third Note (Koretz, 1992) 
establishes a theoretical and procedural framework for evaluating indicators, and a final 
Report will discuss options for improving the monitoring of mathematics and science 
education. 



SUMMARY 



There is growing concern in the United States about the quality of mathematics and 
science education. If this concern is to be translated into educational improvement, then 
information about curriculum is essential. The present study seeks to describe the condition 
of mathematics and science curriculum at the secondary level (including changes in 
curriculum during the 1980s), to evaluate the quality of that description, and to evaluate the 
quality of secondary-level curriculum indicators that could be produced from existing data 
sources. We did not attempt to establish curriculum standards or to judge the quality of the 
curriculum itself; rather we focused on developing indicators that would support such 
endeavors. 

DEFINITION OP CURRICULUM 

For the purposes of this Note, curriculum is defined as the mathematics-specific and 
science-specific features of the educational environment that determine students' 
opportunities to learn. Some features are defined at the classroom level, reflecting the actual 
content and style of instruction; other features are defined at the school, district or state 
levels and are more distant from instruction. The curriculum elements that will be analyzed 
in this Note span this range. They include: graduation requirements, course availability, 
course completion, and course content, including topic coverage, curriculum-specific 
instructional strategies, and instructional equipment and materials. 

DATA AVAILABILITY AND QUALITY 

The amount of data available to describe secondary-level mathematics and science 
curriculum is limited, and information becomes more scarce as one moves from state- or 
district-level features to classroom-level features, i.e., as one moves "closer" to actual 
instruction. In fact, without the addition of data from nonrepresentative studies, it would be 
almost impossible to provide any description of curriculum at the classroom level. 
Furthermore, it is difficult to describe curriculum trends because few of the relevant data are 
collected on a regular basis. 

The data sources that were used in the study are listed in Table S.l. Table S.2 
summarizes the completeness of the curriculum description that can be fashioned from these 
data. 
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Table 8.1 

Sources of Data to Describe Curriculum Features 



Curriculum 



Ongoing Nationally Nationally 



Nonrepresentative 
Data 



Graduation requirements 


CCSSO, ECS, NAEP, 
SASS 


ATS 




Course availability 
Course completion 


NAEP 
NAEP 


NSSME 
HTS 


SRA 


Course content 
Topic coverage 

Course content 
Instructional strategies 

Course content 
Equipment & materials 




NELS 

NAEP-86, NELS, 
NSSME 

NAEP-86, NELS, 
NSSME 


SIMS, SISS, SRA 
SIMS, SISS, SRA 

SIMS, SISS 



Key to Abbreviations: 
ATS 
CCSSO 



Adminiitrator and Teacher Survey— High School and Beyond (1984) 

State Educational Indicators compiled under the auspices of the Council of Chief State School 
Ofiiccra (annual) 

ECS Education Commission of the Sutes survey of minimum hig^i school graduation requirements and 

advanced diploma standards (periodic) 
HTS 1987 Hifi^ School Transcript Study 

lAEP International Aiaesament of Educational Practice (mathematics and science) (1989) 

NAEP National Asaessment of Educational Progress (biennial) 

NAEP-86 The 1985-1986 NAEP included special sections on math and science 

NELS National Education Longitudinal Study ( 1988 eighth grade cohort) 

NSSME National Survey of Science and Mathematics Education (1986) 

SASS School and Staffing Survey (biennial) 

SIMS Second International Mathematics Study (1982) 

SISS Second lEA Science Study (1982) 

SRA School Reform Asiestnoent project (1989) 



Table 

Completeness of Curriculum Description Developed from Available Data* 



Curriculum 
Feature 



Ongoing Nationally Nationally 
Representative Data Representative Data 



Nonrepresentative 
Data 



Graduation requirements 

Course availability 

Course completion 

Course content 
Topic coverage 

Course content 
Instructional strategies 

Course content 
Equipment & materials 



HIGH 

MODERATE 
LOW 



MODERATE 
HIGH p 
HIGH 

LOW 
LOW 



MODERATE 



LOW 

MODERATE^ 
MODERATE^ 
LOW 



•HIGH indicates that the curriculum featuiB can be described thoroughly; MODERATE indicates that some, 
but not most, aspecU can be described; LOW indicates that only a few aspects can be described. 
+For grade eight mathematics and precalculua/calcului only. 
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RESULTS 

Study findings relating to graduation requirements, course availability, course 
completion, and course content are summarized in the following sections. Recommendations 
regarding the selection of indicators and potential data sources are summarized in Table 3.3. 

Graduation Roquirements 

Regular state-level high school graduation requirements in mathematics and science 
increased markedly between 1980 and 1985 to approximately two years of coursework in 
each subject; these requirements have remained relatively stable since then. On the other 
hand, an increasing number of states (16 as of 1989) have advanced graduation standards for 
voluntaiy academically enriched diplomas, which require approximately one additional year 
of coursework each in mathematics and science. 



Table &3 

Recommended Indicator Types and Potential Data Sources 



Curriculum 


Types of Indicators 


Potential Data 


Feature 


Recommended 


Sources* 


Graduation 


Years of math and science required for regular 


ECS 


requirements 


diploma. 

Years of math and science required for 
advanced diploma. 




Course availability 


Percent of schools offering individual math and 
science courses. 

Percent of schools offering combinations of 
college-preparatory courses. 


NAEP (expanded) 


Course completion 


Percent of students completing individual math 


New transcript 




and science courses. 


studies; NAEP 




Percent of students completing combinations of 


(expanded and 




college-preparatory courses. 


validated) 


Course content 


Research necessary to develop and validate 


New data sources 


Topic coverage 


indicators. 




Course Content 


Research necessary to develop and validate 


New data sources 


Instructional strategies 


indicators. 


NAEP (expanded); 


Course Content 


Research nccessrry to develop and validate 


Equipment & materials 


measures reflecting the existence of, quality of, 
access to, or use of instructional resources. 


new data sources 



^Data collection must be designed to permit oompariaons of curriculum opportunities between groups of 
students and schools. 

Key to Abbreviations: 

ECS Education Commission of the States survey of minimum high school graduation requirements and 
advanced diploma standards (periodic) 

NAEP National Assessment of Educational Progress (biennial) 
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The growing use of alternative graduation standards means that at least two 
indicators of graduation requirements should be included in a comprehensive indicator 
system: one focusing on regular coursework requirements in mathematics and science, the 
other on standards for voluntary advanced diplomas* Two such measures would provide a 
reasonably valid indication of the status of state standards as presently constituted; neither 
is likely to be corrupted if included in a national indicator system. However, it would be a 
mistake to place too much emphasis on measures of graduation requirements as key 
indicators of curriculum; they describe only minimum conditions and do so in only the most 
general terms. They do not portray the distribution of courses or course taking across schools 
and students, nor do they differentiate between courses based on title, level or rigor. 

Course Availability 

With minor exceptions, course availability has changed little over the past decade. 
Basic and intermediate college-preparatory mathematics and science courses are available in 
over 90 percent of all high schools, as is at least one course in computer literacy or computer 
science. On the other hand, advanced courses, particularly advanced mathematics courses, 
are unavailable to large numbers of students. Physics, the most common advanced science 
course, is xmavailable in 20 percent of all high schools, while advanced mathematics courses 
(such as calculus) are not available in 25 percent of all high schools. Life science, earth 
science, physical science, and statistics are the exceptions to the trend of stable course 
availability; each of these courses is available currently in one and one-half to two times as 
many high schools as a decade ago. 

AltJiough all schools are likely to offer at least one section of basic and intermediate 
mathematics and science courses, there are substantial differences between schools in the 
availability of advanced mathematics and science courses. Students attending urban schools 
with the lowest parent-occupation profiles (i.e., a small percentage of parents employed in 
professional or managerial jobs and a large percentage of parents unemployed or on welfare) 
are up to five times less likely than students attending urban schools with the highest 
parent-occupation profiles (i.e», a large percentage of parents employed in professional or 
managerial jobs and a small percentage unemployed or on welfare) to be offered advanced 
mathematics and science courses. Similarly, students attending small schools are two to 
three times less likely than students attending large schools to be offered such courses, and 
students in schools with a high percentage of minority students are one and one-half times 
less likely to be offered advanced courses than students in schools with a low percentage of 
minority students. Furthermore, these between-school differences in course availability 
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increase as the level of the course increases, with the greatest differences occurring in 
college-equivalent courses, such as Advanced Placement courses. 

In order to adequately describe course availability, at least two indicators are 
recommended, one focusing on the availability of individual mathematics and science 
courses, the other on the availability of combinations of college-equivalent courses in 
mathematics and science (e.g., geometry and trigonometry; biology, chemistry and physics; 
etc.). In each case it is important to be able to analyze the data by school and student 
characteristics to highlight differential access to courses. 

Unfortunately, current data sources will have to be modified to support such 
indicators. Moreover, additional research on the validity of course availability indicators is 
needed. For example, there is evidence that courses with the same title can differ markedly 
in content. There also is some concern that measures of course availability will be comapted 
in a high-stakes indicator environment, e.g., if schools offer advanced courses of lower quality 
in an attempt to improve their standing with respect to this indicator. 

Course Completion 

The migority of high school graduates complete a core of basic and intermediate 
matliematics and science courses. In contrast, fewer than one-quarter of high school 
graduates complete advanced mathematics and science courses, although advanced courses 
are available in the msgority of schools. 

Course completion rates have been increasing for almost all mathematics and science 
courses, and the increases are proportionally greatest for intermediate and advanced courses. 
For example, the percent of graduates who completed trigonometry, analysis/precalculus, 
chemistry, or physics increased roughly 50 percent between 1982 and 1987. Some of the 
increase in course completion rates may be due to the marked increase in graduation 
requirements between 1982 and 1987. Another factor may be the growing emphasis on 
mathematics and science as critical tools in an increasingly technological world. 

Unfortunately, differences in course completion rates among population groups are 
large and have been growing. Asian and white students complete intermediate mathematics 
and science courses at a rate two to three times greater than black and Hispanic students, 
and the differences increase as the level of the course increases. Differences are greatest in 
mathematics, where course completion rates have been growing 50 percent to 100 percent 
faster for white students than for Hispanic or black students, and 100 percent to 400 percent 
faster for Asian students than for Hispanic or black students. The same differences in 
completion rates between population groups are found in science, but changes over time have 



been somewhat more unifcfin. The only completion rate difference that is increasing is the 
gap between Asian students and all others. 

In contrast, males and females complete most math and sc:.ence courses in equal 
ratios. The only gender-related diflferences in course completion occur in advanced science 
courses: approximately 30 percent more males than females complete three years of science, 
including physics. 

There also are associations between course taking (self-reported by students in 11th 
grade) and school characteristics. For example, 11th grade students who attend urban 
schools with the highest parent-occupation profiles and schools with few students receiving 
subsidized lunches are two to three times more likely than their counterparts in schools with 
the opposite characteristics to complete intermediate mathematics and science courses. 
Similarly, 11th grade students in larger schools, in low-minority schools, and in Catholic and 
private schools are somewhat more likely to complete intenriediate mathematics and science 
courses than students in small schools, high-minority schools and pubUc schools. Moreover, 
these differences in course taking are larger than the corresponding differences in course 
availability, and therefore cannot be explained completely by them. In general, the 
relationships between course taking and school characteristics are stronger in mathematics 
thaui in science. 

T^vo indicators of course completion are recommended, one focusing on completion of 
individual mathematics and science courses and one on completion of combinations of courses 
that characterize the college-preparatory sequence (e.g., geometry atid trigonometiy; biology, 
chemistry and physics, etc.). In each case it is important to be able to analyze the data by 
school and student characteristics to highlight differential course completion patterns. 

The same concerns raised regarding the validity of course availability indicators apply 
to indicators of course completion. In particular, measures of course taking based on student 
self-reports may be subject to corruption if used as elements of a national indicator system; 
transcript-based indicators would be more robust. 

Course Content 

Very few aspects of course content can be described adequately using existing data. 
Most importantly, Uttle is known about the actual topics that are covered or the manner in 
which they are presented. 

The only aspects of course content that can be described even marginally adequately 
are the availabiUty and use of instructional resources, such as textbooks, laboratory facilities, 
computers, and calculators. For example, a small number of different textbooks are used 
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almost universally in mathematics and science classes. However, this coxninonality of 
textbooks does not translate into a uniformity of content^ because teachers do not cover the 
same proportion of the textbooks. 

Computers or computer terminals are available in almost all schools, and the number 
of computers grew rapidly during the preceding decade. However, the ratio of computers to 
students is si:ill lew (approximately 1 to 30 in 1986). Only slightly more than half of all 
schools have enough computers in one location for a full classroovn to use them, and fewer 
than 30 percent of mathematics and science teachers have computers that are readily 
accessible to them. Finally, lesa than 10 percent of student computer use is for the study of 
mathematics or science. 

Similarly, although most schools have calculators (the percentage of schools with 
calculators has grown from 77 percent to 94 percent in the past decade), a minority of 
students actually use calculators in mathematics or science classes. 

Instructional resources are not evenly distributed across schools and classrooms. For 
example, laboratory facilities of one type or another are available in almost three-quarters of 
all high schools; however, specialized science laboratories of the type associated with 
advanced science courses are found twice as often in larger schools than in smaller ones and 
in urban schools with high parent-occupation profiles than in urban schools with low parent- 
occupation profiles. Similarly, schools with relatively high parent-occupation profiles, low- 
minority schools, and non-inner city schools have more computers, calculators, and other 
instructional resources available than other schools. 

At least one indicator of instructional materials should be included in a comprehensive 
curriculum indicator system. Such an indicator might be defined on the basis of the 
existence of selected resources, the quality of the resources, the ease of access to these 
resources for teachers or students (whoever is the predominate user of the resources), or the 
actual use of the resources. Additional research is needed to explore th/^ role of such 
equipment in instruc^^ion and the validity of alternative measures before a specific indicator 
can be recommended. 

In contrast, there are no ongoing, nationally representative sources of information 
about topic coverage or instructional strategies. Considerable research would be necessary to 
define and validate appropriate measures of these aspects of course content and to decide 
which measures to incorporate into an indicator system. Furthermore, course content 
measures (which often are based on self-reports) are particularly susceptible to corruption in 
a high-stakes environment. Caution must be exercised to ensure that such measures are 
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defined and collected in ways that accurately depict the construct of interest rather than 
merely influence reports about it. 

CONCLUSIONS 

Unfortunately, it is not possible to construct an adequate system of curriculum 
indicators based on existing data sources. The indicator patchwork pieced together from 
existing sources is incomplete or uneven in four important ways. First, the patchwork does 
not cover many important aspects of curriculum; i.e., there are significant gaps in our ability 
to describe opportimities to learn mathematics and science. Second, the patchwork is 
temporally uneven. Some data are current, others are almost a decade old; some are updated 
biennially, some quadrennially, and others are unlikely to be updated for a decade. Third, 
the patchwork is of uneven quality. To fill in some gaps it is necessary to rely on data from 
less well-implemented or less rigorous 5?urveys; other gaps cannot be filled at all. Fourth, 
measures drawn fi-om different data sources are not ahv?.ys congruent, so it may not be 
possible to draw desired comparisons. 

As a result, our knowledge of the status of mathematics and science curriculum in U.S. 
secondary schools is inadequate for effective policy making. The greatest gaps in our 
knowledge of curriculum occur at the classroom level: little is known about the actual content 
of courses or the manner in which content is presented. At the present time, a patchwork 
indicator system can provide a picture of curriculum that is adequate only if we are willing to 
ignore such classroom-level variation. 

The lack of information about course content is a serious deficiency. Measures of 
course completion do not adequately reflect students' exposure to specific mathematical and 
scientific knowledge and patterns of thought, nor do they reveal the full extent of differences 
in curriculum opportunities. This limits the value of these measures as a monitoring tool. 
Furthermore, current reform efforts focus on the content and process of mathematics and 
science education. The curriculum patchwork that can be assembled from existing data is 
insensitive to changes likely to be engendered by such reforms, so the patchwork is of limited 
value for monitoring the effects of these efforts. 

The partial picture that can be portrayed from existing data reveals that some 
students have access to broader curriculum opportunities than others, and these differences 
are not random variations but systematic differences associated with identifiable conditions. 
It is critical that such differences be more closely monitored through a mechanism such as an 
indicator system so that problems can be identified, addressed, and, hopefully, alleviated. 
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RECOMMENDATIONS 

Although this analysis revealed serious deficiencies in our ability to describe 
mathematics and science curriculum and to monitor curriculum changes, most of these 
deficiencies can be remedied through additional research and data collection. There are four 
broad areas of action the National Science Foundation (NSF) might consider to fill the gaps 
in the existing data network and to build a comprehensive curriculum indicator system. 

First, although existing sources provide basic data about graduation standards, course 
availability, and course completion, there are no assurances that these data will continue to 
be available on a regular basis in the future. It is likely that data on the first two of these 
features — ^graduation requirements and course availability — ^will continue to be collected (by 
the National Assessment of Educational Progress (NAEP), the Education Commission of the 
States (ECS), the Coimcil of Chief State School Officers (CCSSO), or other agencies), but this 
is not a certainty. Neither the National Assessment Governing Board nor the leadership of 
the Education Commission of the States has specific reasons to collect these data in a style 
and format appropriate for curriculum indicators. It might be prudent to act to ensure the 
continued availability of these two types of core data in an appropriate format. 

Course completion measures pose a greater problem because complete data are not 
collected in any ongoing surveys. Only incomplete self-reported measures of course taking 
are available through NAEP. There are two ways this deficiency might be remedied — ^new 
transcript studies or modifications to NAEP. The most reUable course completion data come 
firom transcript studies, and the most satisfactory solution would be to take actions to ensure 
that regular transcript studies were conducted. As an alternative, NAEP course-taking 
measures could be expanded to provide a basis for course completion indicators. This would 
require extending the range of courses on which students were asked to report. It also would 
require supplemental research to validate these self-reported data against transcript-ba'^ed 
results, because it would be unwise to rely on modified NAEP course-taking measures 
without such validation research. 

Second, the greatest gap in current curriculum data concerns course content. A 
comprehensive curriculum indicator system should be able to describe the content of 
mathematics and science courses and how this content is presented to students. To 
accomplish this, measures of course content would have to be developed, validated, and 
incorporated into ongoing data-collection efforts. NSF already has funded promising 
research to investigate alternative coursework indicators at the eighth grade level, but much 
more research is needed to complete this developmental work and to broaden the scope to 
include subject matter content and instructional strategies at multiple grades. 
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Third, an indicator system reqvdres ongoing maintenance; it is not enough merely to 
collect data and compute indicators. Supplemental research is needed to validate and extend 
the information provided by ongoing data sources. Such research would include validation of 
specific indicator alternatives, examination of the relationship between curriculum measures 
and other student outcomes, and targeted studies of specific topics of interest within and 
across mathematics and science curriculum domains. The maintenance of an indicator 
system requires an ongoing commitment of resources for such supplemental development 
and validation research. 

Finally, more information is needed to describe the mathematics and science 
curriculum at the elementary and middle school levels. It is clear ft*om the Second 
International Mathematics Study (SIMS) and the National Education Longitudinal Study 
(NELS) that curriculum differentiation has begun already by the eighth grade; it would be 
valuable to understand much more about the presentation of mathematics and science prior 
to that grade level. Few sources exist to describe elementary mathematics and science 
curriculum, so much work would have to be done to fill this gap. It would be necessary to 
develop surveys to gather relevant data and to define and validate curriculum indicators 
based on these data. 

The National Science Foundation has a number of options regarding curriculum 
indicators, from fully fiinding all four of the efforts described above to taking no actions at 
all. While all four components would be necessary to have an ongoing, comprehensive, and 
valid mathematics and science curriculum indicator system, they are not equally important. 
Moreover, the actions suggested above do not have equal priorities for NSF. 

It is likely that most of the desired information about graduation standards and course 
availability will continue to be collected by other organizations. These are the two areas of 
curriculum where the potential to build a valid indicator patchwork is the greatest. 
However, small changes and modifications to the w«)rk of ECS, NAEP, and CCSSO would 
increase the value of these data for use in an indicator system. NSF might try to influence or 
coordinate the design of these surveys so they better meet the needs of an indicator system. 

Producing appropriate course completion data may require a somewhat larger effort 
on the part of NSF. Although an expansion of NAEP course-taking measures conceivably 
could be accomplished at Httle cost, it would have to be accompanied by validation research. 
Transcript studies are the preferred alternative, but such studies are expensive. However, 
since transcript studies provide data of value to many educational constituencies, it might be 
possible to develop a collaborative arrangement among educational agencies to share the 
costs of regular studies of this type. 
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The most significant gap in our ability to describe mathematics and science curriculum 
involves course content. NSF has already recognized the importance of this problem, and it 
has sponsored research to explore the development of coursework measures. These efforts 
satisfy a need that is largely unmet through other sources, and the agency should consider 
continuing or even expanding this work until the potential for such indicators is better 
understood. 

Another area in which NSFs efforts may yield significant returns is the ongoing 
enhancement and validation of the basic curriculum indicators derived from other surveys. 
No other group is actively supporting this important work. Such research is necessary to 
maintain the qxiality of the existing incomplete patchwork, to validate additional secondary- 
level curriculum indicators, and to provide information about other issues of interest within 
and across mathematics and science curriculum domains. 

Once a broad secondary-level curriculum indicator system is operational, attention 
should turn to curriculimi opportunities in earlier grades. Existing data suggest that it is 
important to monitor students' exposure to mathematics and science at the elementary and 
middle school levels. However, by postponing development of elementary indicators for a 
time, future work on this topic can benefit from the research done at the secondary level. 
Resources might be used more efficiently as a result. A study to investigate the costs and 
feasibility of developing elementary and middle school curriculum indicators in mathematics 
and science might be a reasonable first step toward a more complete set of elementary and 
middle school curriculum indicators. 
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1. INTRODUCTION 



Why should anyone be interested in educational indicators, particularly indicators of 
curriculum in mathematics and science? The short answer is that the ability to describe the 
status of the educational system is a critical element in efforts to improve it, and that the 
opportimity to learn mathematics and science, i,e,, the curriculimi, is one of the most 
important determinants of student achievement in these subjects. The long answer, which 
fills the next page or two, supplements this explanation with a description of why this issue 
is important at the present time and how this study relates to other attempts to describe the 
educational system. 

Over the past few years, there has been growing concern about the effectiveness of the 
educational system in the United States. Educators are questioning the effectiveness of a 
system that has been marked by declining or static scores on college entrance e. laminations, 
increasing numbers of drop-outs from high school, and increasing doubts about the 
qualifications of teachers. Policymakers are questioning whether schools are producing the 
skilled work force necessary to be competitive in the 21st century. 

Amid this climate of concern, the president and state governors held an educational 
summit in 1989 and established national goals for education for the year 2000. Reminiscent 
of the post-Sputnik goal of being the first to the moon, the assembled politicians declared 
that students in the United States should be first in the world in mathematics and science 
achievement by the year 2000. With these lofty aspirations in place, educators and 
policymakers now are confronted with the daunting task of achieving them. 

How can educational indicators contribute to this process? The chief function 
indicators can perform is to monitor our progress toward achieving these goals. Modem 
engines are built with internal sensors to monitor the status of critical components, and 
mechanics interpret the information provided by these sensors to tune engines to achieve 
maximum performance. It is hoped that educational indicators can be developed to serve a 
similar function, i.e., to describe the performance of students, teachers, and schools, in a 
manner that is accurate and of value to policymakers. 

If there had not already been an effort under way to develop national indicators of 
education, then the ratification of the president's goals would certainly have sparked such 
development. Fortunately, people have been concerned about measuring the performance of 
the educational system for over a decade (Hamischfeger and Wiley, 1975; Breland, 1976; 
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Koretz, 1986; Mullis and Jenkins, 1990), and much work has already been done to describe 
important aspects of schooling, particularly student achievement. 

Unfortunately, measures of student performance such as standardized achievement 
tests, the SAT and ACT examinations, and the National Assessment of Educational Progress 
(NAEP) offer little guidance for educational improvement. To use a metaphor from athletics, 
achievement data do a reasonable job of telling the score (we were falling behind throughout 
the 1970s and early 1980s but began to stabilize and even improve in the latter part of the 
1980s), but they tell us nothing about how to improve it^ Policymakers need information 
similar to that contained in playbooks and game highlight films, e.g., descriptions of 
strategies and successful performances that show how more points can be scored. 

Over the past five years, there have been a number of efforts (including some 
sponsored by the National Science Foundation [NSF]) to develop methods for monitoring the 
status of the educational system as a whole (Raizen and Jones, 1985; Blank, 1986; Gilford, 
1987). Most of these activities focused on the development of a small set of indicators that 
could be used to track key features of education (Kaagan and Coley, 1989; Mumane and 
Raizen, 1988; the Council of Chief State School Officers [CCSSO] 1987). The most 
comprehensive efforts used explicit models of schooling as the basis for selecting critical 
elements for inclusion in the monitoring system (Shavelson et al., 1987). This approach 
increases the likelihood that the information provided by the indicators will be relevant to 
school improvement.^ 

Measures of curriculum were essential elements in many of these models (Mumane 
and Raizen, 1988; Cakes, 1989; McDonnell et al, 1990) because curriculum is a key element 
of schooling and one of the chief determinants of student outcomes. Furthermore, the 
distribution of curriculum is an important measure of educational equity (Oakes et al, 1990). 
A reasonable portrayal of the educational process requires some attention to the curriculum. 

DEFINIK^G CURRICULUM 

What do we mean by curriculum? For the purposcii of this Note, curriculum refers to 
the mathematics-specific and science-specific features of the educational environment that 
determine students' opportunities to learn. Some of these features are determined at the 
district or state level, including minimum graduation standards and other requirements that 
guide students' choices of courses. 

^For additional analysis of the appropriateness of achievement scores for *^lling the score," see 
Koretz, 1992. 

^One of the challenges in developing an indicator system is to balance comprehensiveness with 
practicality, i.e., to identify a small set of statistics that describe important educational constructs. 
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Other features are determined at the school level. These include the syllabi and 
textbooks that help to define the content of instruction. Also included are other enabling 
conditions that affect students' opportimity to learn mathematics and science, such as the 
courses that are offered and the instructional resources that are made available. 

Finally, many aspects of curriculum are determined at the classroom level. Classroom 
features include the traditional notion of curriculum as the substance of instruction, i.e.> 
what is taught. This means the specific topics in the domains of mathematics and science 
that are presented to students. The classroom-level features also include certain content- 
specific aspects of how material is presented, such as the teacher's choice of representations 
to use to operationalize particular concepts, the teacher's goals for student learning in 
mathematics and science, the materials or equipment used to illuminate information, etc. 
We consider these instructional variables to be elements of curriculum because they directly 
affect students' opportunity to learn specific content. (Excluded from this definition are more 
general pedagogical variables, such as pacing, management style, etc.) Together these 
classroom-level features can be thought of as defining course content. 

In combination these state-, district-, school-, and classroom-level features interact to 
create the mathematics- and science-specific environment in which student learning takes 
place. This environment includes: what's required, what's available, what's elected, what's 
presented, how it's presented, and how it's supported. One may think of it as a continuum of 
curriculum influences, ranging from those ''close to" classroom instruction to those that affect 
student opportunities only at a "distance," and we will use this metaphor as a way of 
organizing the presentation of data about curriculum in subsequent chapters. 

It also is helpful when talking about curriculum to distinguish between official policy, 
actual practice, and the impact of curriculum on students. Cross white et al. (1386) 
differentiated between the intended curriculum (defined at the system level by course 
offerings, course outlines, syllabi, and textbooks), the implemented curriculum (defined at 
the school and classroom levels by the courses elected by students, the equipment and 
materials available, and the specific topics and approaches chosen by teachers), and the 
attained curriculum (defined at the student level by the body of knowledge and attitudes 
imparted to students). This Note is concerned with the first two aspects: curriculum policy 
and curriculum practice. Koretz is preparing a companion Note that explores the effects of 
schooling on student achievement. 

Due to the limitations of the available data, we are not able to explore all of the 
aspects of curriculum others have identified as important For example, there is little or no 
information about the scientific and mathematical accuracy of the curriculum or its 



pedagogical quality (Murnane and Raizen, 1988). Similarly, it is not possible to conduct a 
thorough examination of the depth of the curriculum or the sequence of presentation (Oakes 
and Carey, 1989). Finally, existing data provide no basis for monitoring broader elements of 
the school context, such as press for achievement, that affect students' coursework choices 
and performance (Oakes, 1989). 

We use available data to describe four distinct aspects of mathematics and science 
curriculum. These elements range from broad state-level policies (such as minimum 
graduation standards) to specific classroom-level features that define course content (such as 
topic coverage, instructional strategies, and materials). Four important curriculum elements 
will be explored in subsequent chapters: 

• Graduation requirements 

• Course availability 

• Course completion 

• Course content, including 

— Topic coverage 

— Curriculum-specific instructional strategies (e.g., mathematical or scientific 
goals, choice of representation, etc.) 

— Instructional equipment and materials 

RESEARCH QUESTIONS 

The specific goals of this study were to describe the currc^nt conditions of mathematics 
and science curriculum at the secondary level and to evaluate the quality of that description. 
First, a patchwork of curriculum statistics was drawn from existing data sources to portray 
the current status of secondary mathematics and science curriculum (Shavelson et al., 1987). 
Second, the soundness of the data was evaluated, the major gaps in the description of 
secondary curriculum drawn from the data sources were identified, and recommendations 
were made for changes in data collection. Third, alternative indicators of secondary 
curriculum were evaluated, and recommendations were made for adopting a small number of 
indicators and for conducting additional research on indicator development. We did not 
attempt to judge the quality of the curriculum nor to establish curriculum standards; rather, 
we focused on constructing a database that would support these endeavors and others. 

This study addressed the following specific research questions regarding mathematics 
and science curriculum at the secondary level: 
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• VTiat is the status of the curriculum? Specifically, how has curriculum changed 
ovv:;^ time; how evenly is the curriculum distributed across schools and students? 

• Are reliable, representative data available on a regular basis to describe the 
curriculum? 

• What are the strengths and weaknesses of alternative curriculum indicators? 
Are they valid for the intended purposes and resistant to corruption? 

The Currem Status of Secondary Mathematics and Science Curriculum 

At the broadest level, a system of curriculum indicators should answer the following 
questions: What is the nature of the educational opportunities offered to students in 
mathematics and science? What mathematical and scientific content (knowledge, skills, and 
experiences) is presented to students and how is it presented? For the purposes of this 
presentation, these general questions have been translated into two specific research 
questions, one focusing on curriculum trends, the other focusing on the distribution of 
curriculum across policy-relevant groups of schools or students. For each curriculum 
construct presented in subsequent chapters, informatioii will be presented to answer these 
questions. 

Whenever possible, the answers to these questions will be based on current, reliable, 
nationally representative data. When such information does not exist, results from 
exploratory studies may be used to suggest the current status of that particular aspect of 
curriculum. 

Data Limitations 

Unfortunately, there are both practical and theoretical constraints on the description 
of secondary mathematics and science curriculum. On the practical level, existing data 
sources simply do not provide a complete picture of the curriculum. The amount of 
information declines as the focus changes from the curriculum as intended (in the form of 
graduation requirements and course titles) to the curriculum as implemented (in the form of 
topic coverage, instructional strategies, and materials).^ Moreover, the data that are 
available to describe curriculum are of uneven quality; some are quite reliable, valid for use 
in the manner in'-^nded, and resistant to corruption; others are less so. On the theoretical 
level, there is no widely accepted framework for describing some aspects of the secondary 

•^The problem was worse at the elementary and middle school level. So few data were available 
that it was impoesible to include elementary and middle school curriculum in this study. 
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mathematics and science cumculum, notably course content. (This may account, in part, for 
the paucity of data on course content.) 

As a result, the indicator patchwork that can be assembled provides an incomplete and 
inconsistent answer to basic questions about curriculum. One of the purposes of this study 
was to examine how adequate existing data are to support an indicator system. 

Primary Data Sources 

The number of nationally representative studies that assess aspects of secondary-level 
mathematics and science curriculum is quite small (and the number of ongoing studies that 
provide regular updates is even smaller). In fact, it is not possible to bmld an adequate 
picture of curriculum without drawing upon results from nonrepresentative studies. Even 
with the inclusion of large-scale nonrepresentative studies, the nizmber of independent 
sources of information regarding curriculum is small; only a dozen data sources were used to 
generate the vast majority of the results reported in this Note. Table 1.1 lists each of these 
studies and classifies them with respect to curriculum features, representativeness and 
regularity. The studies are described in detail in Appendix A. Table 1.2 summarizes the 
completeness of tfie curriculum description that can be produced using these data. 

Evaluating Indicators 

Another of the major tasks of this research was to evaluate potential indicators of 
secondary mathematics and science curriculum. Traditionally, educational measures (tests, 
inventories, etc.), are evaluated in terms of reliability and validity, and these criteria are 
appropriate for evaluating indicators as well. Reliability is the degree to which a measure is 
free from random measurement error. A measure is reliable if it produces a consistent 
result — either internal consistency (one part with another) or consistency across scorers, 
administrations, or other conditions of measurement. Reliability can be established by 
determining the invariance of measures across different conditions of measurement. 

Validity is the degree to which a measure supports the inferences being drawn from it. 
Measures are not valid in and of themselves; they are only valid with respect to a particular 
inference. Depending upon the inference being made, validity might be established, like 
reliability, by examining the consistency of results. It might also be established by comparing 
the content of the measure with the content of the domain it purports to represent, 
comparing the pattern of results among measures of similar and dissimilar constructs, and/or 
comparing results of the measure with results of some external criterion whose validity is not 
in question. In the present context, a useful indicator of validity is the invariance of results 
across alternative measures and alternative conditions of measurement. 
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Table 1.1 

Sources of Data to Dascribe Curriculum Features 



Curriculum 
Feature 



Ongoing^ationally 
Imitative Data 



Nationally 
Representative Data 



Nonrepresentative 
Data 



Graduation requirements 

(bourse availability 

Course completion 

Course content 
Topic coverage 

Course content 
Instructional strategies 

Course content 
Equipment & materials 



CCSSO, ECS, NAEP, 
SASS 

NAEP 

NAEP 



ATS 

NSSME 

HTS 

NELS 

NAEP-86,NELS, 
NSSME 

NAEP-86, NELS, 
NSSME 



SRA 

SIMS, SISS, SRA 
SIMS, SISS, SRA 
SIMS, SISS 



Key to Abbreviations: 

ATS AdminiBtnitor and Teacher Survey— -High School and Beyond ( 1984) 

CCSSO State Educational Indicaton compiled under the auspices of the Council of Chief State Sdiool 
Officers (annual) 

ECS Education Conuniadon of the States survey of minimum hi^ school graduation requirements and 

advanced diploma standards (periodic) 
HTS 1987 High School Transcript Study 

lAEP International Assessment of Educational Practice (mathematics and science) (1989) 

NAEP National Assessment of Educational Progress (biennial) 

NAEP-86 The 1985-1986 NAEP included special sections on math and science 

NEUS National Education Longitudinal Study (1988 eif^th grade cohort) 

NSSME National Survey of Science and Mathematics Education (1986) 

SASS School and Stafiing Survey (biennial) 

SIMS Second International Mathematics Study (1982) 

SISS Second lEA Science Study (1982) 

SRA School Reform Assessment project ( 1989) 



Table 1^ 

Completeness of Curriculum Description Developed from Available Data* 



Curriculum 
Feature 



Ongoing Nationally Nationally Nonrepresentative 

Representative Data Representative Data Data 



Graduation requirements 

Course availability 

Course completion 

Course content 
Topic coverage 

Course content 
Instructional strategies 

Course content 
Equipment & materials 



HIGH 

MODERATE 
LOW 



MODERATE 
HIGH 
HIGH 
LOW 

LOW 

MODERATE 



LOW 

MODERATE* 
MODERATE* 
LOW 



♦HIG7I indicates that the curriculum feature can be described thorou^ly; MODERATE indicates that some, 
but uot most, aspects can be described; LOW indicates that only a few aspects can be described. 
fFor grsde ei^t mathematics and precalcul us/calculus only. 
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Koretz (1992) reviews likely threats to reliability and validity of indicators and 
concludes that testing the robustness of the results is a key element in assessing both 
criteria. This includes asking whether the results are consistent across data sources (e.g., 
NAEP versus National Survey of Science and Mathematics Education [NSSME]) and 
alternative measures (com-ses available versus courses selected). It also relates to the 
question of corruptibility, i.e., whether measures continue to support the same inferences in 
different contexts (e.g., whether scores improve in a high-stakes environment, such as a 
national indicator system, even though the underlying constructs may not have improved). 

We used these general criteria as bases for evaluating alternative formulations of 
indicators. Particular attention was paid to the potential validity of measures (including 
their resistance to corruption) in an indicator environment. 

ORGANIZATION OF THIS NOTE 

Each of the four major aspects of curriculum is addressed in a separate chapter, 
beginning with graduation requirements. The sequence of chapters is arranged in a "top 
down" order, from more distal state- and district-level features of curriculum to more 
proximal school- and classroom-level features. Each chapter describes the current status of 
one aspect of the secondary curriculum, provides a more detailed analysis of the availability 
and quality of data to support indicators of that aspect of the curriculum, and evaliiates 
alternative formulations for indicators. Particular attention is paid to the likelihood that 
alternative indicators would be corrupted in a high-stakes context. The final chapter 
summarizes the results of the study and offers recommendations regarding the development 
of an effective curriculum indicator system. 

This Note is part of a larger effort to assess the status of precoUege mathematics and 
science education and evaluate the efficacy of a patchwork indicator system. The other parts 
of the study address the status of student achievement in mathematics and science and 
appropriate measwes of reliability and validity in the context of educational indicators 
(Koretz, 1992). The final report of the project will examine the overall condition of 
mathematics and science education across these three areas and evaluate indicator options. 



2. GRADUATION REQUIREMENTS 



This chapter addresses three issues: the current status of graduation requirements in 
mathematics and science, the availability and quaHty of data to describe these requirements, 
and the strengths and weaknesses of alternative graduation requirement indicators. Each 
section begins with a statement of the fundamental question that guided the investigation 
followed by a detailed presentation of relevant information. 

CURRENT STATUS 

What course requirements in mathematics and science must students complete to earn a 
high school diploma, and how have these requirements changed over time?^ 

Two types of graduation requirements have been adopted by state and local boards of 
education: standard requirements that apply to all students, and supplemental or advanced 
requirements that apply to a subset of students who elect to pursue advanced study. These 
requirements are described in the following sections. 

Rdgular Graduation Requirements 

At least 45 states establish specific minimum requirements for high school graduation. 
In the remainder of the states, the authority for setting graduation standards has been 
delegated to local boards of education (or it resides with them constitutionally). With the 
exception of these five states, state-level standards define the minimum educational 
requirements for a regular high school diploma. In all cases, the basic requirements describe 
conditions that must be met during a student's last four years of schooling, grades 9 throixgh 
12. 

Table 2.1 shows the student-level average number of years of mathematics and science 
courses required for graduation. These values rose substantially between 1974 and 1985 and 
have remained essentially unchanged since then. 

State requirements, although they are the most prominent, are not the only 
graduation standards that apply to students. In many instances, state standards are 
supplemented by district and/or school requirements, increasing the cumulative graduation 
standards that must be met by students. In theory, one should be able to determine 
gradiiation requirements enacted at any administrative level: state-level, district-level, 

^Graduation requirements apply equally to all students within a jurisdiction, so there is no need 
to examine the distribution across difTerent groups of students. 
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Table2.1 

State*ImpoMd Regular Graduation Requirements in 
Mathematics and Science 





Mean Number of Years Required by Typical Student* 


Subject 


1974 1980 1983 1985 1987 1989 1990 


Mathematics 
Science 


.74 87 1.6 2.0 2.1 2.1 2.1 
.81 .84 1.4 1.7 1.8 1.8 1.8 



♦ Based upon total K-12 enrollments reported in 1976, 1981, 1986 and 1989. 
SOURCES: National Association of Secondary School Principals, 1975 & 1980; 
Education Commission of the States, 1983 & 1990; Council of Chief SUte School OfTicers, 
1987 & 1990; Center for Education Statistics, 1987; National Center for Education Statistics, 
1989; Clune, 1989. 



schooHevel or cunmlative. In practice, the only data that are available from existing sources 
are those that reflect cumulative requirements across all levels. 

Graduation requirements established by local boards and schools in the absence of (or 
as a supplement to) state requirements exceed somewhat those promulgated by states. Table 
2.2 shows the school-level (in some cases district-level) average number of years of 
mathematics and science required to moet all state and local graduation requirements. 
These values have been increasing gradually over time, although changes in the level at 
which the data were collected may account for some of these differences. 

Advanced Graduation Standards 

A number of states have established standards for advanced diplomas in addition to 
their regular graduation standards. There are various designations for these programs, 
including ''advanced studies," "academic scholars," "college-preparatory studies," etc. The 
advanced standards differ from the regular ones in many ways, including the number of 
reqmred courses in particuJar subjects, the type and level of required courses, the number of 

Table 2^ 

Cumulative Regular Graduation Requirements in 
Mathematica and Science 
(As Reported by Schools and Districts) 

Mean Number of Years Required by Schools 

Subject 1982 1984 1986 1987* 1988* 

Mathematics 1.7 1.9 1.6 2.3 2.4 

Science 1.5 1.6 1.4 2.0 2.1 

♦Mean number of years required by districts. 

SOURCES: RAND tabulations of data from the 1985-1986 National 
Asses ment of Educational Progress, RAND tabulations of data from the 
1987- 988 School And Staffing Survey. 
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electives, the establishment of mimmum grade point averages, and the use of qualifying 
examinations. The number of states enacting such standards grew from 6 to 16 between 
1985 and 1990. Furthermore, in 14 of the 16 states, the new requirements included 
additional coursework or examinations in mathematics or science. The average (across 14 
states) number of years of mathematics and science courses required for advanced 
graduation is displayed in Table 2.3, and the mean difference (across 14 states) between 
advanced and regular graduation standards over time is shown in Table 2.4. 

AVAILABILITY AND RELIABILITY OF DATA 

Are reliable, representative data available on a regular basis to describe high school 
graduation requirements? 

Data Availability 

Until recently, there was no ongoing national survey of state graduation requirements. 
Data for the analysis of regular graduation requirements were drawn from a number of 

Table 2^ 

State-Imposed Advanced Graduation Requirements in Mathemati<^ft and Science 



Mean Number of Years Required 





1974 


1980 


1983 


1985 


1987 


1990 


Subject 


(3 states) 


(2 states) 


(2 states) 


(6 states) 


(13 states) 


(14 states) 


Mathematics 


1.7 


1.5 


2.0 


2.8 


3.1 


3.1 


Science 


1.3 


1.5 


1.6 


2.5 


2.9 


2.9 



SOURCES: RAND tabulations of data from the Center for Education Statistics, 1987; National 
Center for Education Statistics, 1989; the National Association of Secondary School Principals, 1975 & 
1980; and the Education Commission of the SUtes, 1983 & 1990. 



Table 2.4 

Difference Between State-Imposed Begular and Advanced Graduation Sequirements 

in Mathematics and Science 



Mean Number of Years Required 





1974 


1980 


1983 


1985 


1987 


1990 


Subject 


(3 states) 


(2 states) 


(2 states) 


(6 states) 


(13 states) 


(14 states) 


Mathematics 


1.0 


0.5 


1.0 


0.7 


0.9 


0.9 


Science 


0.3 


0.5 


0.5 


0.5 


0.9 


0.8 



SOURCES: RAND tabulations of data from the Center for Education Statistics, 1987; National 
Center for Education Statistics, 1989; the National Association of Secondary School Principals, 1976 & 
1980; and the Education Comnoission of the SUtes, 1983 & 1990. 
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difFerent sources ^ and data regarding advanced diploma standards came from CCSSO (1990) 
and the Education Commission of the States (ECS, 1990). 

The irregular and uncoordinated nature of the data collection efforts raised questions 
about the availability of information in the future. An unofficial, catch-as-catch-can 
approach does not provide the ongoing flow of data needed to support an indicator system. 
Fortunately, it appears that one organization, the ECS, has assumed the responsibility for 
collecting and updating these data on a periodic basis.^ (In recent years, data on advanced 
state diploma standards also have been included in the database.) This provides some of the 
continuity that was lacking when no one group took responsibility for collecting information 
regarding state graduation requirements. However, since no federal funds are provided to 
support the collection of these data, their continued availability depends upon the interests 
and resources of an independent organization. Furthermore, it will continue to be difficult to 
assess data quality without direct monitoring of data collection and analysis or independent 
verification of results. 

Data for generating estimates of cumulative regular graduation requirements (state-, 
district- and school-level requirements combined) were collected by three national school- 
level surveys: the 1984 High School and Beyond Administrator and Teacher Survey (ATS), 
the 1985-1986 National Assessment of Educational Progress (NAEP), and the 1988 School 
and Staffing Survey (SASS). Unfortunately, as Table 2.2 revealed, the three sources do not 
paint a consistent picture of cumulative requirements; the 1986 NAEP results are 
inconsistent with the trend portrayed by the other sources (and with the trend in state-level 
requirements portrayed in Table 2.1). Furthermore, the 1987-1988 SASS results were 
collected at the district rather than the school level, which may in part account for 
differences between these data and the data from the other surveys. Taken together, the 
differences between the results raise some doubts about the reliability of the surveys, 
primarily the reliability of the data collected by NAEP. 

Which data are likely to be available in the future? It is unlikely that ATS will be 
repeated. On the other hand, both NAEP and SASS are intended to be biennial surveys. 
Although the content of these surveys is not fixed, the surveys could include regular 

^The National Association of Secondary School Principals (1974 and 1980), the Education 
Comnusaion of the States (1983 and 1990), the Center for Education Statistics (1987), the National 
Center for Education Statistics (NCES, 1989), The Council of Chief State School Officers (1987 and 
1990) and Clune (1989). The Clune report, in turn, drew upon the previously cited work of ECS and 
NCES as well as Goertz (1988) and Belsches-Simmons et al. (1987, March). 

^ECS data files on state graduation requirements are updated whenever changes are enacted. 
However, summary reports are issued only when a ^substantial number of changes have occurred since 
the last report." (Personal communication.) 
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assessment of cumulative graduation requirements, if sufficient interest is shown in these 
results. (It should also be possible to improve the question formats to test our assumptions 
about the accuracy of the results.) 

At the present time, neither NAEP nor SASS collect^; data on advanced graduation 
requirements. While it would be possible to include this topic in future surveys, the extra 
effort does not appear warranted. To our knowledge, advanced standards are almost 
exclusively a state-level phenomenon. (There is no information about the extent of locally 
defined advanced graduation standards.) Consequently, it seems to make sense to rely on 
state-level reports for this information. 

Data Reliability 

Because so many researchers collected data on state graduation requirements, there 
was ample opportunity to assess the consistency of results across data sources. Although 
there were a number of small discrepancies between concurrent results, most were 
procedural in nature (e.g., different operational definitions). Tlie number of errors in data 
collection or reporting was small. 

The number of procedural discrepancies in tabulating or analyzing regular 
requirements was slightly greater, but not great enough to cause alarm. Furthermore, most 
could be resolved easily. For example, some researchers included new requirements in the 
year they were enacted, and others included them in the year they became effective for 
graduating seniors. It should be fairly simple to standardize operational definitions in the 
future so changes in requirements are reported consistently. The discrepancies most difficult 
to resolve arose from differences in the way researchers treated the complex options or 
substitutions permitted in some state requirements. For example, if students were required 
to complete two years of mathematics, two years of science, and a fifth year of either subject, 
there was little consistency in how this was reported. Furthermore, as options grew more 
complex, there were greater differences in reporting. Requirements that include choices will 
continue to be a problem for data reporting, and they may continue to complicate or confound 
the computation of indicators. 

There was a relatively greater proportion of discrepancies in the reporting of advanced 
standards than in the reporting of regular standards. It would appear that most differences 
were due to a lack of standardization in defining and labeling criteria, but this was harder to 
verify on the basis of the results themselves. 
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EVALUATION OF INDICATORS 

What are the strengths and weaknesses of alternative indicators of high school 
graduation requirements? Are they valid for the intended purposes and resistant to 
corruption? 

Regular State Graduation Requirements 

The distribution of graduation requirements across states can be used to compute two 
different indicators: the average value across the 45 states portrays the requirements in 
effect in a typical state (regardless of the number of students enrolled in high school); the 
weighted average (based on the number of students in the state) portrays the minimum state 
standards that must be fulfilled by a typical student in the United States. In theory, these 
two averages could differ considerably if large states and small states had different 
graduation requirements. In fact, since 1983, they have differed by no more than five 
percentage points. This similarity occurred because there was no systematic relationship 
between total enrollment and state graduation requirements, and it is unlikely that this 
situation will change in the future. As a result, the two statistics should continue to describe 
the same trend, and either could serve as an adequate indicator of graduation requirements. 

However, the two measures have different meanings. The choice of which to use as an 
indicator depends upon the intended use. The advantage of the across-student average is 
that it permits direct comparison with other measures of schooling that can be reported at 
the student level. For example. Tables 1.1 and 1.2 can be used to compare state 
requirements with cumulative requirements, which were available only at the student level. 
In a similar manner, one might look at average graduation rates, post-secondary educational 
enrollments, or achievement scores during the same period of time to see if relationships 
existed between these and graduation standards. 

The cross-state average, in comparison, would be more meaningful if one were 
interested in educational variables occurring at the state level, such as teacher-licensing 
standards, and state support for education. We recommend the former definition (and we 
reported results in this format in the previous section) because the student-level perspective 
offered the greatest commonality across different aspects of curriculum. 

Finally, there is a subtle problem of definition that plagues both indicators. The 
problem arises because not all states establish specific graduation standards. The absence of 
state requirements may mean different things, which might lead one to treat them 
differently in computing an overall average value. Some states set standards that are low (or 
even zero), while others do not set standards at all, but leave this responsibility for local 
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boards of education. It is difficult to know whether to exclude a state when computing the 
mean (on the grounds that there are no state requirements), include it (on the grounds that 
the state sets requirements but its requirement in mathematics or science is zero) or impute 
a value (based on an estimate of the averioge). The choice should depend upon the intended 
use of the data. 

Furthermore, some of the states that leave standard setting to local school boards 
strongly recommend minimum course requirements for the boards to adopt, which becomes 
the de facto state standard. In this report we have included all states in the computation; 
those that had no minimum requirements in mathematics or science (or only made 
recommendations) were counted as zero. The statistic produced in this manner reflects (more 
or less accurately) the actions of states (though it clearly underestimates the cumulative 
graduation requirements affecting students)* 

One consequence of including zero requirements in the calculation of the state-level 
average is that recent increases in the value of this statistic are, in part, artifacts of changes 
in the agency that sets graduation requirements, not the level at which they are set. The 
number of states with no state standards in mathematics and science declined from 14 in 
1980 to 6 in 1989. Because of the way these states were treated in defining the indicator, the 
change from local to state requirements will increase the value of the indicator. 
Unfortunately, there is no information about the degree to which these new state 
requirements differed from existing local ones. As a result, it is impossible to determine 
what proportion of the change in the value of the indicator was due to the substitution of 
state standards for local ones of the same rigor and what proportion was due to an increase 
in the overall requirements faced by students. 

Once a decision is made regarding the treatment of zero coursework requirements, 
both the cross-state and cross-student averages can be computed unambiguously, and both 
offer a number of strengths as indicators. First, either indicator is interpretable in everyday 
terms: students must take mathematics and science classes two of the four years they are in 
high school to satisfy regular state standards for graduation. Second, either indicator lends 
itself easily to longitudinal comparisons. For example, regular state graduation 
requirements in mathematics and science increased dramatically between 1980 and 1985 but 
have remained relatively stable since 1985. Third, the indicators are relatively stable, i.e., 
changes in the value of the indicator reflect real changes in state policy regarding graduation 
standards. (For the purpose of comparison, the distribution of regular state graduation 
standards in mathematics and science is shown in Tables C.l and C.2 in Appendix C.) 
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On the other hand, both formulations have potential weaknesses. First, the mean of 
graduation requirements across states or across students is not sensitive to small changes in 
requirements that might be significant from a policy perspective. For example, a change in 
course requirements in one state from two years of mathematics or science to three years 
may reflect a hard-fought change in attitudes among policymakers, but it will have little or 
no impact on the value of the indicator. Neither the mean nor the standard deviation would 
be affected to a degree that would indicate the significance of the change in state curriculum 
practices. The full distribution of state requirements in mathematics and science would 
reflect such changes more accurately, although it would be unwieldy as an indicator. 

Cumu!atlve Regular Graduation Requirements 

In many instances, state standards are supplemented by district and/or school 
requirements, increasing the cumulative graduation standards that must be met by students. 
An indicator could be defined in terms of cumiilative standards, and (with the exception of 
1986) it would depict a trend similar to that shown by an indicator based on state-level 
standards. (See Tables 2.1 and 2.2.) 

We recommend against the use of a single indicator based on cumulative graduation 
requirements for three reasons. First, data to describe state-level requirements are available^ 
more readily. Second, these data appear to be more reliable than the data that describe 
cumulative requirements. Third, state-level requirements are more directly subject to policy 
intervention. 

However, if the standard-setting context changes and local authorities assume greater 
responsibilily for setting graduation requirements, an indicator based on state requirements 
will no longer provide a valid picture of minimum graduation standards. In these 
circumstances, an indicator based on cumulative requirements would be preferable. If 
adequate resources are available, it would be useful to track both sets of requirements and 
monitor the degree to which local authorities find it necessary to supplement state actions. 

Advanced Graduation Standards 

Some indication of the status of advanced diploma standards is important, because it 
may help explain trends that are observed in other features of schooling. For example, the 
rapid increase in the establishment of advanced standards in the mid-1980s may partially 
explain the stability in regular state graduation requirements during this time period: 
policymakers focused their attention on a different mechanism to enhance curriculum 
standards for a selected group of students, those intent on college. 
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What is the best measure to use to monitor the status of advanced graduation 
standards? A "head count" of the number of states adopting advanced graduation 
requirements that affect mathematics or science is one option. However, this approach 
reveals Uttle about the nature of the requirements, which vary dramatically from state to 
state. For example, one state requires two additional years of coursework in both 
mathematics and science while another imposes no increase in coursework, but adds a 
comprehensive examination. A tally provides no insight into the rigor of the supplemental 
standards. 

In contrast, the average number of years of mathematics and science coursework 
required by the states for an advanced diploma provides a more accurate picture of the 
intensity of these programs. Unfortunately, once the distribution of standards is known, an 
average can be computed in three different ways — based on the number of states, the total 
number of students in states, or the number of students who elect to pursue advanced 
diplomas. Furthermore, one could compute an average based on the absolute level of the 
standards or an average based on the difference between advanced and regular standards. 

We recommend the state^evel average for two practical reasons. First, no information 
is available at present about student participation rates, so the latter mean cannot be 
computed. Second, the fact that there are two different student-level averages is likely to 
create confusion about the meaning of either of these two indicators. 

However, this definition has weaknesses. One of the drawbacks of an advanced- 
standards indicator defined as a state-level average is that it may be inappropriately 
compared to other noncomparable statistics, particularly indicators of regular standards. By 
necessity, the state-level average for advanced requirements is based on a subset of states 
(14 at present), while the state-level average for regular requirements is based on the entire 
country. As a result, it is incorrect to draw inferences about mean differences between 
regular and advanced standards based on the two averages, even though they are both 
defined at the state level. 

For example, Table 2.3 shows that the typical state advanced graduation standard in 
1990 was 3.1 years of mathematics and 2.9 years of science (based on 14 states). At the same 
time, the typical state regular graduation requirement was 2. 1 years of mathematics and 1.8 
years of science (based on 60 states).^ However, it was not true that the increase in advanced 
standards over regular ones was the difference between these two statistics (1.0 years of 

'♦These are the correct values of the state-level averages. They were not reported in Table 2.1 
because they differed little from the student-level averages. 
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matheinatics and 1.1 years of science). Similarly, it would be incorrect to use any two of the 
three values to compute the other value. 

A different indicator would be needed to monitor differences between supplemental 
requirements and regular requirements. The natural way to define such an indicator would 
be the mean of the differences between advanced and regular standards in those states that 
have adopted both types of standards. In 1990, 14 states required 0.9 additional years of 
mathematics and 0.8 additional years of science, on average, to meet honors or college- 
preparatory studies requirements. (See Table 2.4.) 

Although this indicator overcomes the problem of inexact comparisons, it suffers from 
shortcomings of its own. First, by providing only relative information, the indicator conveys 
nothing about the absolute level of the standards. An increase in mathematics requirements 
from one to two years would appear equivalent to an increase from two to three years. In 
some instances one might want to equate these two changes (they both might reflect similar 
tightening of state policies), while in other circumstances they should be differentiated 
(students with advanced diplomas in one state might be less well prepared than students 
with advanced diplomas in another). 

Second, and more problematic, the value of the relative change indicator decreases as 
regizlar standards increase. This may convey the false impression that advanced standards 
are declining, when in fact they remain the same. For example, Rhode Island increased its 
regular requirements from one year each of mathematics and science courses to two years 
each without increasing its requirements for an advanced diploma. As a result, the 
difference between regular and advanced standards decreased. 

Which statistic is more meaningful, the absolute level of advanced standards or the 
relative difference between advanced and regular standards? As the previous discussion 
illustrated, the two indicators are more meaningful in combination than either is alone. Both 
may be required to accurately track advanced graduation standards. However, if a single 
indicator must be chosen (for reasons of cost or simplicity) we would recommend an indicator 
based on absolute level rather than one based on relative differences. 

Another important consideration is the degree to which the proposed indicators are 
corruptible in different political contexts. If the stakes associated with performance as 
measured by these indicators increased (i.e., if greater attention were paid to the number of 
years of mathematics and science courses students were required to complete to receive a 
high school diploma), states might feel pressure to raise absolute requirements. Whether 
this is a good policy is debatable. We know that increased state graduation requirements in 
the mid-1980s resulted almost exclusively in increases in the number of basic, general, or 
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remedial courses offered (Clune, 1989). We can aesume that a similar phenomenon would 
occur if additional pressure were put on states in terms of regular graduation requirements. 
However, it is not possible to improve one's standing with respect to this indicator without 
making a real change in standards. To this extent the indicators are not corruptible. 

Would an indicator of cumulative requirements be equally resistant to corruption, i.e., 
would local respondents exaggerate graduation requirements in an effort to portray their 
program as more rigorous? Since data are collected at random, and results are not 
associated with individual respondents, this is highly unlikely. 

On the other hand, there may be some negative consequences of using either indicator 
(average regular state graduation requirements or average cumulative graduation 
requirements) as an element of a national indicator system. The pressure to increase the 
total ntimber of years required might work to the detriment of states, such as Oklahoma, 
that have made standards more rigorous in terms of curriculum and course level but have 
relaxed the absolute number of courses required. Greater attention to the number of courses 
required would run counter to this type of meaningful educational reform. 

Further Limitations of Graduation-Standards Indicators 

An additional shortcoming of all the graduation-standards indicators (and all 
indicators based on years of coursework) is that they equate all courses in a subject, (e.g., 
counting General Science the same as Advanced Placement Chemistry, Consumer 
Mathematics the same as Trigonometry). As a result, the indicators reveal little about the 
mathematics and science courses students actually take or the content to which students are 
exposed. For example, two students who graduate from high school having completed the 
same number of years of mathematics may have taken dramatically different courses. A 
vocational student might take courses that focused on applied computation and algebra while 
a college-preparatory student might take courses in statistics and calculus. Bot students 
could satisfy the graduation requirements with educational experiences that were completely 
dissimilar. 

This insensitivity to differences in course level is not a deficiency in the definition of 
the indicator but a limitation in the way states have chosen to implement policies regarding 
graduation. For the most part, states have chosen to ignore differences in the level of courses 
when establishing graduation standards, opting instead to define requirements in terms of 
years of coursework with no reference to course content or difficulty. 

While the indicators described here accurately reflect the nature of most state 
requirements, we still might wish for more refined information. Knowing that two students 
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completed the same number of years of mathematics provides some information about their 
preparation, but it offers little insight into the more fundamental question, What level of 
mathematics and science instruction are students exposed to in high school? Course 
completion and course content data are much more revealing about the implemented 
curriculum than minimum requirements. 

It is worth noting that at least one state has requirements that are sensitive to 
differences in the level or difficulty of courses, Oklahoma offers an advanced, college- 
preparatory diploma whose standards are both more rigorous and more flexible than those in 
place for a normal diploma. Rather than requiring a greater number of courses, the state 
substituted more demanding standards regarding the choice of courses. The result is that 
the number of courses required for the college-preparatory diploma (15) is less than the 
number required for regular graduation (20). In this instance, an indicator based solely on 
the number of years of coursework would be misleading, giving the impression that the 
college-preparatory program was less rigorous than the regular one. 

As this example shows, the validity of an indicator based on average years of 
coursework would drop dramatically if more states adopted standards that focused on the 
content of the curriculum and the level of the course while relaxing requirements regarding 
the number of courses. An alternative indicator that distinguished between course 
requirements based on level or difficulty would be required to portray this situation 
accurately. Information about the level or difficulty of course requirements might be of great 
value to policymakers; however, most states do not express graduation requirements in these 
terms. The indicators of graduation requirements proposed here will not provide an 
indication of course level. Nevertheless, it appears that a coursework average will provide a 
valid picture of minimum graduation standards for the near future. 
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3. COURSE AVAILABIUTY 



Course availability refers to the opportunity schools present to students to enroll in 
(and as a result to master the content of) specific courses in mathematics and science. This 
chapter begins with information about the current availability of courses and the distribution 
of course offerings across schools. It continues with an analysis of the quality of the data 
that are available to describe course availability, and then concludes with an evaluation of 
alternative indicators relative to this topic. 

CURRENT STATUS 

How widely available are secondary-level mathematics and science courses and how 
hc^ course availability changed over time? 

Courses are the basic organizational units of curriculum in secondary schools. 
Subject-matter knowledge and skills are clustered into coherent sequences designed to be 
taught over a period of one semester or one school year. Courses are designated by titles, 
and mathematics and science titles are quite similar throughout the country (Weiss, 1987). 
This similarity makes it possible to develop various indicators of secondary mathematics and 
science curricula based on course titles. 

There are limitations to the use of courses as the basic unit of analysis, which are 
discussed below in the section on the validity of course-availability indicators. However^ 
because courses are the organizing unit of the secondaiy curriculum, it is difficult to conceive 
of an indicator system that does not include measures of course availability or course 
completion. Appropriate caution must be used in interpreting measures based on course 
titles. 

Tables 3.1, 3.2, and 3.3 display the percent of high schools that offered common 
mathematics, science, and computer science courses. The tables confirm that almost all 
schools offered algebra and geometry, biology, and chemistry and some type of computer 
science course. Schools did not appear to differ in terms of course availability at the basic 
and intermediate levels, i.e., the courses were fotind in almost all schools. However, higher 
level courses were not universally available, and, as will be shown below, lack of availability 
was associated Math specific school and student characteristics. 

One set of courses of particular interest to those concerned about the production of 
future mathematicians and scientist^ are the highly advanced mathematics and science 
courses that cover curriculum comparable to that included in credit courses at four-year 
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Table 3.1 
Mathenatici Course Availability 

Percent of Schools 



Course Title Offering Course* 



General mathematics, grade 9 64 

General mathematics, grade 10 46 

Business mathematics 49 

Consumer mathematics 49 

Remedial mathematics 29 

Pre-algebra^ntroduction to algebra 63 

Algebra, first year 99 

Algebra, second year 92 

Geometry 95 

Trigonometry 59 

Probability/statistics 14 

Advanced sr. math, without calculus 36 

Advanced sr. math, with calculus 34 

Calculus 31 

Advanced placement calculus 18 

Any calculus or advanced mathematics 76 



*A11 schools with iir&des 10 to 12. 
SOURCE: Weiss, 1987. 



Table 3^ 
Science Course Availability 



Course Tide 


Percent of Schools 
Offering Course* 


Ldfe science 


46 


Earth science 


52 


Physical science 


68 


General science, grade 9 


31 


General science, grades 10-12 


18 


Biology, first year 


99 


Chemistry, first year 


91 


Physics, first year 


81 


Biology, second year 


53 


Chemistry, second year 


28 


Physics, second year 


11 


Astronomy 


8 


Anatomy/physiology 


32 


Ecology, environmental science 


15 


Zoology 


6 



*A11 sdiools with grades 10 to 12. 
SOURCE: Weils, 1987. 



colleges. This set, referred to as ''college-equivalent courses," includes Advanced Placement 
courses in mathematics, science, and computer science; second-year courses in biology, 
chemistry, physics, and computer programming; and calculus courses. Table 3.4 shows that 
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TableS^ 

Computer Science Coune Availability 





rerceut oi ochouib 


Course Title 


Offering Course* 


Computer awareness of literacy 


62 


Applications and implications of 


24 


computers 


65 


Introductory computer programming 


Advanced computer programming 


38 


Advanced placement computer sci^^nce 


13 


Any computer science 


91 


♦All schooU with grades 10 to 12. 




SOURCE: Weiss, 1987. 




Table 3.4 




College^Equivalent Course Availability* 




Percent of Schools 


Subject 


Offering Course 


Mathematics 


73 


Biology 


60 


Chemistry 


33 


Physics 


18 


Computer science 


46 



^Advanced Placement or second year courses 
SOURCE: RAND tabulations of daU from the 198&-1986 
National Assessment of Educational Progress. 



the percentage of schools that offered college-equivalent courses ranged from 73 percent in 
mathematics to only 18 percent in physics. 

What was the trend in course availability over the past decade? Table 3.5 shows only 
a modest increase Qess than or equal to five percentage points) in the availability of common 
mathematics and science courses between 1977 and 1986. The lack of growth in the 
availability of basic and intermediate courses can be explained, in part, by the high 
percentage of schools that offered these courses. Geometry, algebra II, biology I, and 
chemistry I were offered in over 90 percent of all high schools, so little increase was possible. 
(It is disappointing to see that the percentage of schools ofiTering geometry actually declined 
between 1977 and 1986.) On the other hand, the percentage of schools offering advanced 
courses, such as trigonometry , calculus, and physics I, also changed little between 1977 and 
1986, despite the fact that there was ample room for improvement 

The greatest growth in course availability occurred in the sciences; a substantial 
percentage of schools began offering life science, earth science, and physical science for the 
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Table 3 

Comparison of Common Mathematics and Science 
Courae Availability: 1977 and 1986 



Percent of Schools* 
Offering Course 



Coursp Title 


1977 


1986 


Mathematics 






General math, grade 9 


59 


64 


General math, grades 10 to 12 


42 


46 


Business mathematics 


D/ 


AQ 


Geometry 


97 


95 


Algebra n (advanced algebra) 


87 


92 


Trigonometry 


54 


59 


Probability/statistics 


7 


14 


Calculus 


31 


31 


Science 






Life science 


18 


46 


Earth science 


37 


52 


Physical science 


40 


68 


General science, grade 9 


46 


31 


General science, grades 10-12 


11 


18 


Biology, first year 


95 


99 


Chemistry, first year 


89 


91 


Physics, first year 


81 


81 



♦All schools with grades 10 to 12. 
SOURCES: Weiss, 1978; Weiss, 1987. 



first time between 1977 and 1986. This appears to represent a change in the traditional 
science curriculum that should be monitored carefiilly in future surveys. 

How evenly is course availability distributed across secondary schools and students? 
Nationwide course availability statistics, such as those presented in the previous section, 
provide an incomplete picture of the curriculum. They fail to portray between-school 
differences in course availability associated with certain characteristics of the schools and 
their students. An indicator system would be deficient if it did not provide information that 
could be used to monitor these differences. 

Three benchmark courses from each field were selected to use as comparative 
measures: 

Mathematics Science Computer Science 

algebra II physics computer literacy 

trigonometry biology II programming I 

precalculus/calculus Advanced Placement programming II 

(AP) biology 
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These courses were selected for two reasons. First, little differentiation was likely to 
be found among introductory courses available in over 90 percent of all schools. Second, 
NAEP (which provided the best information on student and school characteristics) did not 
collect course availability data on introductory courses, focusing only on intermediate and 
advanced courses. 

For these analyses, schools were compared on five dimensions: size (measured as the 
average enrollment per grade level), size and type of commtmity (a variable constructed by 
NAEP to reflect a combination of location and parent occupation^), lunch participation 
(defined as the percent of students participating in the subsidized lunch or nutrition 
program), minority enrollment (defined as the percent of non-Asian minority students in the 
school), and type (public, private, or Catholic). 

To analyze the distribution of course availability, we used a technique that focused on 
differences between groups of schools. Schools were clustered into groups based on the 
characteristic of interest, and wi thin-group percentages of schools offering each course were 
computed. For example, schools were subdivided into quartiles based on total enrollment, 
and the percent of schools in each quartile group offering each course was computed. (See 
Figure 3.1.) As Figure 3.1 illustrates, the proportion of schools that offered selected 
mathematics courses increased as school size increased. (For comparable data regarding the 
relationship between course availability and school size in science and computer science 
courses, see Figs. C.l and C.2 in Appendix C.) 

By way of comparison. Figure 3.2 shows mathematics course availability by size and 
type of community. Here the greatest differences occurred between high metropolitan and 
low metropolitan schools, although the differences changed depending upon the level of the 
courses. High metropolitan and extremely rural schools offered intermediate courses in 
approximately equal percentages. However, high metropolitan schools were more than twice 
as likely as low metropolitan or extremely rural schools to offer advanced mathematics 
courses. 

These group statistics can be used to calculate indicators of differentiation. We have 
chosen to use a comparison between the mean values of the largest and smallest schools as a 
measure of the severity of the differentiation of course availability associated with school 
size. In particular, the ratio of mean percentages in the extreme groups provides a numerical 

^Three extreme subgroups were compared: High Metropolitan, defined as the top 10 percent of 
urban schools in terms of parent occupation; Low Metropolitan, defined as the bottom 10 percent of 
urban schools in terms of parent occupation; and Extreme Rural, defined as the bottom 10 percent of 
rural schools in terms of parent occupation. 
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■ 1--30 ■H^"'"*^° i 191-240 1 1241 + 
Students HsH students — » students I » students 




Algebra II Trigonometry Calc/precalc 

Course title 

SOURCE: RAND tabulations of the 1985^-1986 NationaJ Assessment of EducationaJ Progress 

Fig. 3.1 — ^Mathematics Course Availability by Grade Level Enrollmeiit 

indication of the extent to which students in tJie largest and smallest schools have similar 
access to selected mathematics courses. 

Table 3.6 contains course availability ratios (between extreme groups) for four factors: 
total enrollment, size and type of community, population groups, and subsidized lunch 
participation. The ratios in the first column of Table 3.6 reflect the degree of difference in 
access to courses based on school size. For example, the ratio of calculus availability for large 
schools versus small schools was 2.2, meaning that the percentage of large schools that 
offered calculus was more than twice as great as the percentage of small schools. This also is 
equivalent to an estimate of the relative probability that large and small schools will offer 
specific courses; e.g., tlie probability that a large school will offer precalculus/calculus was 
over twice the probability of a small school. (Figures depicting course availability in 
mathematics, science, and computer science for all four variables will be found in Appendix 
C, Figures C.3 through C.13.) 

The results depicted in Table 3.6 confirm that curriculum (in the form of course 
availability) was distributed unevenly, based on school and student characteristics (Oakes et 
al, 1990). The table illustrates a number of relationships. First, schools were less likely to 
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SOURCE: RAND tabulations of the 1985-1986 National Assessment ot Educational Progress 



Fig. Mathematics Course Availability by Size and Type of Commimity 



Table 3.6 
Course Availability Ratios 





Large/Small 


High/Low 


Low/High Minority 


Low/High Lunch 


Course Title 


Schools* 


Metropolitan+ 


Enrollment** 


Participation++ 


Mathematics 










Algebra II 


1.1 


1.2 


1.1 


1.1 


Trigonometry 


1.2 


1.9 


1.3 


1.2 


Precalculus/calculus 


2.2 


2.8 


1,6 


2.2 


Science 










Physics I 


1.4 


1.6 


1,2 


1.3 


Biology II 


2.0 


3.5 


1,8 


1.5 


AP biology 


2.6 


5.2 


1.4 


2.5 


Computer Science 










Computer literacy 


1.0 


1.1 


1.1 


1.2 


Programming I 


1.6 


2.0 


1.4 


1,5 


Programming II 


3.6 


4.6 


2.3 


6.2 



•Ratio of firit quartile mean percentage ofFering course to fourth quartile mean percentage offering course. 
♦♦Ratio of 0 to 25 percent non -Asian minority group mean percentage to 75 to lOO percent non-Asian 



minority group mean percentage. 

+Ratio of high metropolitan school mean to low metropolitan school mean. 

-H-Ratio of 0 to 19 percent subsidized lunch group mean to 40-t- percent subsidized lunch group mean. 
SOURCE: RAND tabulations of daU from the 1985-1966 National Aiaeiament of Educational Progress. 
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offer advanced courses if they were small rather than large, had a high rather than low 
percentage of non-Asian minority students, had a high rather than low percentage of 
students receiving subsidized lunch, or drew students from urban neighborhoods with low 
rather than high parent-occupation profiles.^ Second, the degree of difference in course 
availability increased as the level of the course increased. Third, the pattern of 
differentiation was similar for mathematics, science, and computer science, and was most 
severe for computer science. Fourth, the ratios of differentiation were somewhat greater for 
size and type of commtmiiy (STOC) than for the other factors, although this was due, at least 
in part, to the fact that the high and low categories of STOC were smaller in number and had 
less variation than the extreme categories of the other variables. School population groups 
affected course availability somewhat less than the other three characteristics. 

School type (public, private. Catholic) did not affect course avsulability as strongly as 
the four factors mentioned above. The probability that a school offered selected 
mathematics, science and computer science courses was approximately the same regardless 
of whether the school was public or Catholic. However, the proportion of private schools 
offering selected courses, particularly computer science courses, was lower than the other two 
groups. (See Figures C.ll through C.13 in Appendix C.) 

Do all students have equal access to the most advanced courses? The previous 
analysis showed that the answer is no. We found the patterns of differentiation even more 
striking in the case of college-equivalent courses. Students in small schools, urban and rural 
schools with low parent-occupation profiles, high-minority schools, and schools with a high 
percentage of students receiving subsidized lunches were less likely to have access to college- 
equivalent mathematics and science courses than students in large schools, urban schools 
with high occupational profiles, schools with few minority students, and schools with few 
students participating in the school lunch program. For example, schools in which a high 
percentage of students were receiving subsidized lunches were less than half as likely as 
schools in which a moderate percentage of students were receiving subsidized lunches to offer 
collie-equivalent courses in mathematics, chemistry, or physics. (See Figure 3.3. For 
comparable information regarding the other school characteristics, see Figures C.14 through 
C.17 in Appendix C.) 

Measures defined in terms of extreme groups focus on the worst instances of unequal 
access. For this reason, they may be an effective tool for encouraging policymakers to 

^Thia pattern of relationships is not completely surprising, since the dimensions on which 
schools were compared were not independent. For example, urban schools with low parent-occupation 
profiles were generally characterized by high percentages of minority students and high percentages of 
students receiving subsidized lunch. 
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Mathematics Biology Chemistry Physics 

AP or second-year course In subject 

SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 
Fig* 3.3 — College-Equivalent Course Availability by Percent Receiving Subsidized Lunch 

address inequities. Certainly, declines in these course availability ratios would represent 
genuine lessening of differentiation where it is most severe. 

AVAILABIUTY AND RELIABILITY OF DATA 

Are reliable, representative data available on a regular basis to describe secondary-level 
course availability? 

There are two recent nationally representative sources of data regarding course 
offerings in mathematics and science that can be used as the basis for computing an indicator 
of course availability. Both the National Assessment of Educational Progress (NAEP), which 
included mathematics and science in 1985*1986, and the 1985-1986 National Survey of 
Science and Mathematics Education (NSSME) offer information that is relevant to this 
construct NAEP is an ongoing biennial survey; NSSME was designed to be a one-time 
update of the 1977 National Survey of Science, Mathematics, and Social Studies Education 
(Weiss, 1978). To our knowledge, there are no current plans to repeat the NSSME survey in 
the future, so these data cannot be relied upon to meet the needs of an indicator system. 



ERLC 



50 



-30- 



This is unfortunate, for NSSME is, in many respects, the richer of the two databases for 
monitoring curriculum. 

Because there were two concurrent sources of data regarding course av£ulability in 
1985-1986 (NAEP and NSSME), it was possible to test the. consistency of the findings 
directly. Unfortunately, there were a few significant disagreements between the results that 
cast doubts on their reliability. Table 3.7 compares the percentage of schools offering 
mathematics, science, and computer science courses reported by NAEP and NSSME. The 
results in science were reasonably similar, differing by no more than three percentage points. 
However, the results in mathematics and computer science were vastly different. 

Part of the discrepancy can be explained by the fact that the two surveys used 
different labels for some courses. For example, NAEP asked about "precalculus/calculus" 
while NSSME specified only "calculus." However, this does not explain the differences of 50 
percent or more in the values reported for trigonometry, probability/statistics and computer 
literacy. These discrepancies call into question the reliability of the course availability data 
from both surveys. While NSSME appears to have been more conscientious in gathering 

Table 3.7 

Comparison of Mathematics and Science Course Availability 
from Two National Surveys 



Percent of All High Schools 
Offering Course 



Course Title NAEP NSSME 



Mathematics 






Algebra, second year 


94 


92 


Trigonometry 


84 


59 


Probability/statistics 


23 


14 


Calculus 


67* 


31 


AP calculus 


38** 


18 


Science 






Physics, first year 


83 


81 


Physics, second year 


13 


11 


Biology, second year 


56 


53 


Chemistry, second year 


28 


28 


Computer science 






Computer literacy 


89 


62 


Computer programming, first year 


75 


65+ 


Computer programming, second year 


42 


38++ 


AP computer science 


18 


13 



*Precalcu luii/caiculus. 
**AP math. 

+Computer programming, introductory. 
++ Computer programming, advanced. 

SOURCES: Weiss, 1987; RAND tabulations of data from the 1985-1986 



National Assessment of Educational Progress. 
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course-related data, there is no specific basis for discounting the NAEP results. 
Consequently, one must be somewhat dubious about the data on advanced mathematics 
course availability from both sources. 

A direct comparison also points out another problem with NAEP as a source of course 
availability data. The designers of the survey chose to focus the school questionnaire on 
advanced courses, rather than courses at all levels. As a result, no information was gathered 
about the availability of algebra I, geometry, biology I, or chemistry I. This reduces NAEFs 
value as a basis for describing this aspect of the curriculum. Fortunately, it would be easy to 
modify future surveys to include these courses. In fact, the questions on course taking in the 
spiraled Student Questionnaire included a more extensive (although still incomplete) list of 
courses. 

We also have concerns about the quality of the NAEP data, particularly the data on 
course availability, participation in the subsidized school liinch program, and size and type of 
community. Missing data were a serious problem in the cases of the course availability and 
school lunch participation variables. Between 10 percent and 20 percent of the schools failed 
to report data about course offerings and about the percent of students receiving subsidized 
lunches. (See Appendix A for a more complete list of NAEP variables with high rates of 
missing values.) We chose not to eliminate variables from our analysis solely on the basis of 
missing data. Instead, we examined the patterns of missing data and the correlations 
between related variables before making decisions about which variables to include and 
which to exclude from the analysis. For example, further analysis of the characteristics of 
schools that were missing course offering information did not support the hypothesis that 
these schools were less likely than the rest to offer specific courses. Consequently, we 
included this variable in the analysis. 

A different problem affected the use of the size and type of community variable. NAEP 
reported two demographic variables for each school, urbanicity (urban, suburban, rural), 
which was taken from Quality Educational Data files and STOC (size and tjrpe of 
community). STOC is a NAEP-generated variable that reflects a composite of community 
size and location and an occupational profile of parents provided by the school principal. 
Unfortunately, the two were not consistent For example, 24 percent of those schools 
classified as Big City and 32 percent of those classified as Small Place by NAEP were 
classified as suburban by QED, Neither measure appeared to be reliable across all values. 
As a result, we opted to focus on three of the seven levels of STOC reported by NAEP. We 
chose to use only the three extreme values— the 10 percent of urban schools with the highest 
occupational profile (High Metropolitan), the 10 percent of urban schools with the lowest 
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occupational profile (Low Metropolitan) and the 10 percent of rural schools \\dth the lowest 
occupational profile (Extreme Rural). These categories were the most reliable when 
compared to urbanicity and the easiest to interpret. The other STOC categorier did not 
correspond to urbanicity in any predictable manner. (Unfortunately, for an indicator system, 
only NAEP computes and reports size and type of community.) 

Finally, other research points up at least one oversimplification (or oversight) in both ^ 
NAEP and NSSME. The American Institute of Physics found that approximately 72 percent 
of public schools offer physics on an annual basis, but another 18 percent of pubhc schools 
offer it biennially (Czujko and Bernstein, 1989). Both NAEP and NSSME ignored this 
distinction, leaving it up to the respondent to decide whether or not last year's physics course 
should be counted on this year's survey. This may cause a small underestimation in the 
proportion of schools that offer such courses, and the underestimation is most likely to occur 
in small schools, where advanced courses may be offered every other year or irregularly. 

EVALUATION OF INDICATORS 

What are the strengths and weaknesses of alternative indicators of course availability 
at the secondary level? Are they valid for the intended purposes and resistant to corruption? 

This discussion will address five issues relating to the definition and interpretation of 
potential indicators of course availability: the choice between a measure of access and a 
measure of existence, the interpretation of an indicator based on course titles, the universe of 
generalization for course availability statistics, the use of extreme groups and extreme group 
ratios, and the resistance of the indicators to corruption in a high-stakes context. A related 
topic, the identification of a more parsimonious subset of courses for reporting, will be 
discussed in the section on indicators of course completion. 

There are two possible ways to operationalize a school-level indicator of course 
availability: in terms of existence and in terms of access. The former approach addresses the 
question "Which mathematics and science courses were offered?" The latter addresses the 
question "How many sections of each mathematics and science course were otfered?" Both 
approaches provide information about courses; however, they provide different portrayals of 
course availability. Two schools that offer the same course may not be providing equal access 
to that course depending upon the number of sections that are offered, the time of day they 
are offered, and the number of students who want to enroll in the course. 

How does one chose between the two alternatives for defining an indicator of course 
availability? The choice should be based on the meaning of the indicators: Which comes 
closest to our understanding of the underlying construct? The answer to the existence 
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question (Is chemistry offered?) provides an indication of minimum educational opportunities 
offered by the institution. The fact that a course is offered is a necessary, but not sufficient, 
condition for students to gain access to the curriculum covered by the course. The answer to 
the access question (How many sections of chemistry are offered?) provides an indication of 
the extent to which students can and do avail themselves of particular curriculum 
opportunities. The access measure indicates student course-taking behaviors as much or 
more than intended administrative policies. 

We recommend using the dichotomous measure of the existence of courses as an 
indicator of minimum institutional standards, because it comes closest to the construct of 
course availability as we understand it. Student factors influence the existence of courses to 
a lesser degree than they influence the number of sections of a course, so the former measure 
reflects institutional intentions relatively more than the latter. Of course, this distinction is 
somewhat artificial. Administrative policies and student actions are like **a pair of 
intertwined vicious circles" (Neuschatz and Covalt, 1988), and one must not be tricked into 
thinking they can be untwisted just by choosing between two operational definitions. It is 
impossible to disentangle the effects of course taking from course offering and produce a pure 
indicator of either construct. 

If one accepts the recommendation to measure course availabihty in terms of 
existence, one must still exercise caution regarding its interpretation. First, the dichotomous 
indicator does not reflect the frequency with which a particular course is taught, nor the 
proportion of students who are exposed to the content. While geometry and algebra are both 
offered in over 90 percent of the high schools, only 21 percent of all mathematics courses 
taught are geometry, while 36 percent are algebra (Weiss, 1987). The indicator suggests that 
geometry and algebra are equally available: however, they are not equally prevalent. In fact, 
the number of students who take algebra is approximately 60 percent greater than the 
number who take geometry. Second, it would be incorrect to interpret course availability 
indicators as an indication of students' opportunity to learn a particular topic. The existence 
measure provides information about the percentage of schools in which courses are taught at 
least once each year, not the likelihood that a school will provide courses to all qualified or 
interested students. Nevertheless, it is the closest we can come to measuring the 
administrative component of course availability. Furthermore, there is enough variation 
across schools in the existence of courses to make this measure important. 

The second general question that must be addressed is the proper interpretation of 
indicators based on course titles (be they indicators of course availability or course 
completion). What is known about students' opportunity to learn the skills of algebra from 
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the statement that 99 percent of the schools offer algebra I? On one hand, it is true that a 
course called algebra I is taught in almost all schools, so students in almost all schools can 
learn something about algebra (ignoring for the moment the question of students' access to 
the course). On tlie other hand, it is not necessarily true that these courses cover the same 
material. 

While there is some evidence that courses with the same title tend to use the same 
textbooks (suggesting that the intended curriculum is similar), there is considerable evidence 
that actual topic coverage varies tremendously between courses with the same title 
(suggesting that the implemented curriculum is different). McDonnell et al. (1990) found 
that topic coverage varied enormously across sections of algebra i m six different schools. 
Topic coverage varied so dramatically that some students in courses called algebra I were 
receiving no more exposure to algebra topics than some students in general mathematics 
courses. Similar disparities also have been observed in advanced courses, such as physics 
(Neuschatz and Covalt, 1988). To the extent that course titles obscure important differences 
in content, course-availability indicators will provide incorrect impressions about students' 
access to knowledge and skills. 

The one instance in which titles closely match content is the case of Advanced 
Placement courses. These courses are distinguished by a carefully monitored curriculum 
that is established by a national consiilting board. They also have a common set of 
examinations that are used formally to certify student mastery of content and informally to 
allow teachers to monitor their instructionpl focus. The presence of a common curriculum 
and a common examination standardizes the content of Advanced Placement courses to a 
degree not found in any other course. As a result. Advanced Placement courses provide more 
meaningful curriculum milestones than many other courses, and their inclusion in an 
indicator system is encouraged.^ 

The third question concerns the desired universe of generalization for reporting course 
availability statistics. In most cases, the same survey data can be analyzed to answer two or 
more different questions. For example, NAEP data can be analyzed to produce estimates of 
the percentage of schools that offered particular courses or the percentage of 11th grade 



^By noting the value of a standarcdzed curriculum as a basis for meaningful indicators, we do 
not necessarily endorse the particular curriculum embodied in the Advanced Placement syllabus. 
There are other approaches to defining secondary mathematics and science curriculum (American 
Association for the Advancement of Science [AAAS], 1989; National Council of Teachers of Mathematics 
[NCTM], 1989) that might be preferred. Our point is that the commonality of curriculum across AP 
classes is unique in mathematics and science at the present time and offers some advantages for 
developing indicators. 
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studente attending these schools.^ (See Table 3.8.) The choice of which representation to use 
is important because the alternatives do not convey the same impression. For e^-ample, 38 
percent of the schools offer Advanced Placement calculus, but 51 percent of the 11th grade 
students attend these schools. This difference occurs because large schools are more likely to 
offer calculus. As Table 3.8 illustrates, similar effects were evident for most intermediate 
and advanced courses. 

Since the two sets of results are equally reliable (they were based on the same data), 
the choice of which to report must be made on conceptual grounds: which data reflect the 
construct course availability in the most meaningful way? Because the decision to offer 
courses is made by school administrators rather than students, we opted to present results 
regarding course availability at the school level. However, if we were more interested in the 
degree of access to courses, then student-level results might be more informative. Because 

Table 3^ 

Comparison of Alternative Measures of Access to Science and 
Mathematics Courses 



Course Availability 

Percent of 11th 
Percent of High Grade Students 
Schools Offering Attending Those 
Course Title Course Schools 



Mathematics 



Algebra, second year 


94 


99 


Trigonometry 


84 


95 


Probability/statistics 


23 


32 


Precalculus/calculus 


67 


8? 


AP calculus 


38 


51 


Science 






Physics, first year 


83 


96 


Physics, second year 


13 


23 


Biology, second year 


66 


71 


Chemistry, second year 


28 


52 


Computer science 






Computer literacy 


89 


92 


Computer programming, first year 


75 


87 


Computer programming, second year 


42 


60 


AP computer science 


18 


25 



SOURCE: RAND tabulations ofdsU from the 1985*1986 National Asaessment of 
Educational Progreai. 



^NAEP results also can be analyzed to estimate the percentage of secondary students attending 
schools that offer particular courses. However, these results are almost identical to those for 11th 
grade students, so they were not reported here. Differences between these two measures would be 
noteworthy because they would indicate a relationship between course availability and student 
attrition. Fortunately, such differences were not present 
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the differences are substantial, some might argue that both approaches should be included in 
an indicator system. 

Fourth, caution must be exercised in the use of extreme groups and extreme group 
ratios as the basis for indicators. The problem with defining indicators in terms of groups of 
schools is that the definition of the groups can affect the value of the statistics and the 
apparent relationship between the underlying variable and the indicator construct. 
Depending upon the actual distribution of schools on the two measures, it might be possible 
to show dramatically different relationships by merely changing the cut points (in the case of 
ordinal variables) or the selection of levels (in the case of categorical variables). 

To provide consistent indicators, subgroup definitions would have to be standardized. 
Quartiles are a reasonable candidate for a standard, and they were used to define school 
groups based on total enrollment. However, other alternatives should be considered as well. 
In the case of school population groups, where the variable itself was expressed as a 
percentage, groups were defined in terms of the values of the variable (0 to 24 percent non- 
Asian minority, 25 to 49 percent non-Asian minority, etc.). This approach seemed less 
susceptible to misinterpretation. Groups can also be defined in terms of other student 
characteristics, such as achievement. Boundary points on the subsidized lunch variable were 
determined by examining bivariate plots with achievement and dividing the distribution 
based on achievement differences. 

A more extensive study of the relationship between background variables and school 
status measures (including curriculum, achievement, teacher quality, etc.) might provide a 
better basis for clustering schools. Certainly, the overall value of the indicator system would 
be enhanced if the same set of school groups were used to report information about all the 
components of schooling, including curriculum, achievement, teacher quality, etc. 

A number of different statistics could be used to describe variation in course 
availability across particular background variables. One advantage of using extreme groups 
is that a ratio based on values from extreme groups focuses on divergent cases and thereby 
emphasizes discrepancies. An indicator defined in this manner will change little unless 
variation is reduced between conditions at the bottom and the top of the distribution. 

However, ratios of differentiation defined in terms of extreme groups have 
disadvantages as well. The value of a ratio depends upon the definition used for creating the 
groups, and the ratio can be changed by defining extreme groups differently. For example, in 
the case where there is a strong positive relationship between course availability and a 
background variable, the ratio of differentiation can be made larger or smaller by selecting 
wider or narrower tails of the distribution. Another drawback to ratios of differentiation is 
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that the underlying notion of extreme values may have little meaning when the classification 
variables are discrete, as in the case of STOC or school type. In th^se instances, there is no 
natural extreme case, so it would be difficult to select a single ratio that captured the degree 
of nonuniformity. 

Finally, we should consider how course availability indicators are likely to behave as 
the political and educational contexts change. There is some concern that heightened 
attention to the percentage of schools that offer a course will corrupt the meaning of this 
statistic. Although data drawn from a representative sample of schools cannot be associated 
with the actions of any particular school or district (so there is no direct incentive for 
administrators to change the courses they offer), it would be naive to ignore the possibility 
that schools or districts would be held accountable by local officials to the same criteria that 
have been adopted nationally. 

Increased pressure at the local level might lead a school or district to offer courses that 
were not available in the past. If the courses are of high quality and students are adequately 
prepared to take them, this is all to the good. If the instructors are not qualified or the 
students are ill-prepared, the result will be detrimental to students, even though the action 
will improve the school's standing on the indicator of course availability. Furthermore, the 
eventual outcome may be a decrease in the rigor of the courses. Pressures to increase 
requirements could lead to a "watering down" of courses and a change in the meaning of the 
course labels (Clune, 1989). Unfortunately, there is no simple way to mitigate against this 
possibility. A technique for factoring course quality into the indicator system might have the 
desired effect; however, the technology to accomplish this does not yet exist. This is another 
area in which additional research might prove extremely valuable. 
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4. COURSE COMPLETION 



Course completion^ patterns provide an important measure of students* exposure to 
the content of high school science and mathematics. Because course completion is a more 
direct measure of students' actual interaction with mathematics and science than either 
graduation requirements or course availability, measures of course completion deserve 
considerable attention in a secondary curriculum indicator system. This chapter presents 
information about the current status of student course completion, the data sources that are 
available to describe course completion patterns, and the validity of alternative indicators of 
course completion. 

CURRENT STATUS 

What percentage of students completed common secondary-level courses in mathematics 
and science, and has student course completion changed over time? 

Tables 4.1 and 4.2 present course completion results for common college-preparatory 
mathematics and science courses and course combinations. The tables show the majority of 
graduates completed the basic cooirses; about half of the graduates completed the 
intermediate courses; and fewer than 20 percent of high school graduates completed the 
advanced courses. 

Table 4.3 shows the total number of credits^ earned by high school graduates in 
mathematics, science, and computer science. Although the average student earned 
considerably more credits in mathematics and science than were required for regular high 
school graduation, and almost as many as were required for an advanced or college- 
preparatory diploma, the average student did not take advanced, college-preparatory 
courses. They complied with graduation requirements by taking courses at the basic or 

^In this report we distinguished between two related constructs — course completion (receiving 
credit for completing a course with a passing grade) and course taking (enrolling in a course). Some 
surveys (e.g., the 1987 High School IVanscript Study) measure course completion, while others (e.g., 
NAEP) rely on student self-reports of course taking (TTave you ever taken...?*). Conceivably, the latter 
group includes students who did not complete the course or did not receive a passing grade. The size of 
this population is uncertain, although student self-reports of course taking have been shown to be a 
reasonably accinate reflection of course completion in many cases (Valiga, 1986; National Center for 
Education Statistics, 1984). Since the correspondence is not perfect, we opted to differentiate between 
results based on earned credits (course completion) and results based on course enrollment (course 
taking). 

^One unit of credit was defined to be one course taken for one period each day for the complete 
school year (Westat, 1988). 
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Table 4.1 
Mathematics Courses Completed 



Percent of High School Graduates 
Course Title (duration)* 1982 1987 



Individual courses 



Any math 


97 


99 


Any remedial math course/below grade level 


33 


25 


Algebra I 


65 


76 


Geometry 


46 


62 


Algebra II (.5) 


35 


47 


Trigonometry (.5) 


12 


19 


Analysis/precalculus (.5) 


6 


13 


Calculus (all) 


5 


6 


AP calculus 


2 


3 


Statistics/probability (.5) 


0.3 


0.4 


Course combinations 






Algebra II -f geometry 


28 


42 


Algebra II + geometry ••■ trigonometry 


7 


15 


Algebra H + geometry + trigonometry + calculus 


1 


2 



^All courses are st least one year in duration, unless otherwise noted. 
SOURCE: WesUt, Inc., 1988. 



Table 4^ 
Science Courses Completed 



Percent of High School Gradiaates 
Course Title (duration)* 1982 1987 



Individual courses 



Any science 


95 


99 


Biology 


75 


88 


Chemistry 


31 


45 


Physics 


14 


20 


Engineering 


0.1 


0.1 


Astronomy (.5) 


1 


1 


Geology (.5) 


14 


16 


AP/honors biology 


7 


3 


AP/honors chemistry 


3 


3 


AP/honors physics 


1 


2 


Course combinations 






Biology + chemistry 


28 


43 


Biology -1- chemistry -i- physics 


11 


17 



*A11 courses are at least one year in duration, unless otherwise notei. 
SOURCE: WesUt, Inc., 1988. 



intermediate level. In fact, between one-third and one-quarter of all graduates completed 
mathematics courses classified as remedial. 
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Table 4^ 

Credits Earned by High School Graduates 





Mean Number 




of Credits Earned 


Subject 


1982 1987 


Mathematics 


2.5 3.0 


Science 


2.2 2.6 


Computer science 


0.1 0.4 


SOURCE: WesUt, Inc., 1988. 



What were the trends in mathematics and science course completion? The percentage 
of students completing most of the mathematics and science courses listed in Tables 4.1 and 
4.2 increased significantly between 1982 and 1987. Furthermore, the increases were 
greatest, in relative terms, among courses at the intermediate and advanced levels. For 
example, the percentage of graduates who completed trigonometry, analysis/precalculus, 
chemistry, and physics increased roughly 50 percent during that five-year period. An 
accompanying decline occurred in the percent of students completing remedial or below grade 
level courses in mathematics. 

Comparing trends in course completion with trends in course availability suggests that 
the low course completion rates for advanced coursos were not solely the result of lack of 
access to courses- (See Tables 3.1 and 3.2.) Not only was the percentage of schools that 
offered at least one section of advanced mathematics and science courses quite high, but this 
-number did not change very much during the period of rapid growth in the percentage of 
students who completed advanced courses. This suggests that factors other than course 
availability, such as student preparation, guidance counseling, and schoolwide press for 
achievement, were responsible for the increase in student course taking. However, as Oakes 
et al. (1990) point out, course availability may still limit access to advanced courses if schools 
(particularly those serving inner-city, urban neighborhoods) do not offer an adequate number 
of sections to accommodate student demand. 

How evenly is course completion distributed across students and schools? 
Differences in course completion rates in 1987 will be discussed first, beginning with 
population-group differences followed by gender-related differences. Differential changes in 
course completion rates between 1982 and 1987 will be discussed next, also beginning with 
population-group differences and concluding with gender-related differences. Finally, 
associations between school and student characteristics and course taking (as measured by 
NAEP) will be presented. 
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There were significant population-group differences in mathematics and science course 
completion in 1987. (See Tables 4.4 and 4.5.) Asian and white students completed 
intermediate and advanced mathematics and science courses in much greater percentages 
than black and Hispanic students. Furthermore, the relative differences between ihe 
completion rates of the groups increased as the level of the courses increased, particularly 
the difference between Asian students and the other groups. Asian students completed the 
more rigorous mathematics and science sequences at approximately twice the rate of white 
students and approximately four times the rate of black and Hispanic students. 



Table 4.4 

Combinatioiis of Mathematics Courses Completed by Sex and Population Group 







Percent of 1987 High School Graduates 












Geometry, 




Never Taken 




Geometry, 


. Algebra II, 


Group 


Geometry and 


Algebra H, and 


Trigonometry, 


Geometry 


Algebra II 


Trigonometry 


and Calculus 


Sex 










Males 


39 


42 


15 


3 


Females 


38 


43 


14 


2 


Population group 










White 


35 


47 


17 


2 


Black 


56 


29 


8 


1 


Hispanic 


60 


24 


7 


2 


Asian 


19 


62 


31 


15 


SOURCE: RAND tabuUtions of 1987 High School Tranacript Study. 








Table 4«5 






Combinations of Science Courses Completed by Sex and Population Group 






Percent of 1987 High School Graduates 
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Biology I, 
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Biology I and 


Chemistry I, 


Biology I 
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13 


87 


44 
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10 
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42 


13 
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White 


11 


89 


46 


18 


Black 


14 


86 


29 
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15 


85 


28 
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Asian 


8 


92 


66 


42 


SOURCE: RAND tabulations of 1987 High School Tranacript 


Study. 




There were almost no differences in mathematics course completion associated with 


ider, but there were gender-related differences in the completion of advanced science 
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course combinations. (See Tables 4.4 and 4.5.) A far greater percentage of males than 
females completed the three-course sequence: biology, chemistry, and physics. These results 
are consistent with those reported in an earlier study based on data from High School and 
Beyond (West, Miller, and Diodato, 1985). 

Although overall mathematics and science course completion rates increased between 
1982 and 1987, the increases were not uniform across population groups. Figures 4.1 and 4.2 
show that the population-group differences were substantial. Black and Hispanic students 
showed greater gains in the completion of basic mathematics and science courses than did 
white and Asian students (whose completion rates for these courses were already quite high). 
On the other hand, Asian and white students showed greater increases in the completion of 
intermediate math and science courses than did black and Hispanic students. At the highest 
level, the percentage of students who completed the most rigorous college-preparatory 
combinations of mathematics and science courses increased far more among Asian students 
than any other group. The percentage of Asian students who completed 
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Mathematics course combinations 

SOURCE: Westat. Inc.. 1988 

Fig* 4*1— Change in Graduates Completing College-Preparatory Mathematics 
1982-1987 by Population Group 
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Science course combinations 
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SOURCE: Westat. Inc., 1988 

Fig, 4»2— <;hanges in Graduates Completing College-Preparatory Science Courses 

1982-1987 by Population Group 

geometry, algebra II, trigonometry, and calculus increased by more than ten percentage 
points between 1982 and 1987, while the percentage of white, black, and Hispanic students 
completing these courses increased by only one or two percentage points. 

In contrast, the increases from 1982 to 1987 in the percentage of male and female 
graduates completing basic and intermediate mathematics and science courses were 
approximately the same. 

Using the NAEP data, it was possible to compare the course-taking behaviors^ of 11th 
grade students in schools that differed on selected school and student characteristics. (There 
was little evidence about students' completion of advanced courses, such as precalculus or 
calculus, which are normally taken in the 12th grade.) The analysis of mathematics course 
taking was conducted using basic and intermediate mathematics courses (geometry, algebra 

3NAEP provided self-reported data on course taking ("Did you take chemistry?") rather than 
transcnpt-based data on course completion. 
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II, and trigonometry). The analysis of science course taking was based on student enrollment 
in biology I, chemistry I, and physics L* 

Mathematics course taking was strongly associated with school population groups, size 
and type of community, subsidized lunch participation and school type. (See Figures 4.3 
through 4.6.) Students in low-minority schools, high-metropolitan schools, schools with low 
percentages of students receiving subsidized lunch, and private and Catholic schools, were 
more likely to take basic and intermediate mathematics cov ^es by grade 11 than were 
students in schools with the opposite characteristics. There was no association between 
mathematics course taking and enrollment. (See Figure C. 18 in Appendix C.) 
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Algebra II 
Course title 

RAND tabulations of the 1985-1986 National Assessment of EducationaJ Progress 

4.3 — ^IVFathematics Courses Taken Through Grade 11 by Percent Non-Asian 

Minority Enrollment 



^Physics I was included in the analysis (even though it is an advanced course) becaJase 12 
percent of tiie students in the NAEP sample reported they had taken physics. Although it is possible 
that students took physics in the 11th grade, this seems unlikely, and the results regarding physics 
course taking must be interpreted with considerable caution. There was some evidence that students 
responded incorrectly to this question, indicating they had taken physics when they had actually taken 
physical science (an introductory course). 

The analysis of the distribution of science course taking across schools has another limitation. 
There was very little between-school variation in the percent of students taking either biology or 
physics: Almost all students took biology I; very few took physics. As a result, there were almost no 
relationships between 'mckground variables and course taking for these two courses. There was 
greater between-schooi variation in the percent of students taking chemistry, and some associations 
with chemistry course taking were detected. 
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SOURCE: a^ND tabulations of the 1985-1986 National Assessment of Educational Progress 

Fig. 4.4— Mathematics Courses Taken Through Grade 11 by Size and Type of Community 



In general, differences in course taking across schools were more pronounced than 
differences in course availability. For example, Figures 3.2 and 4.4 show that high- 
metropolitan schools and extremely rural schools were equally likely to offer geometry and 
algebra II, but 11th grade students in high-metropolitan schools were far more likely than 
11th grade students in extremely rural schools to take either course. Similarly, there were 
greater differences in course taking than in course availability across schools clustered by 
population groups, subsidized lunch participation, or school type. (See Figures 4.3, 4.5, 4.6, 
C.5, C.8, and C.ll.) 

These comparisons confirm the obvious facts that: (1) offering a course is a necessary, 
but not sufficient, condition for students to take it, (2) offering a course is not the same as 
offering enough sections of a course for all students who wish to take it, and (3) offering a 
course does not, in and of itself, provide adequate preparation and/or incentives for students 
to take it. (Consequently, remedies that focus merely on promoting more widespread offering 
of courses are unlikel: ' to eliminate diflFerences in course taking.) These comparisons also 
provide a strong argument tliat an indicator of course offerings cannot be used as a 
substitute for an indicator of course taking. 
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Fig. 4.5 — ^Mathematics Courses Taken Through Grade 11 by Percent Receiving 

Subsidized Lunch 

Figures 4.7 and 4.8 show the relationship between science course taking and size and 
type of community and school type. These differences were similar in size to those for 
matJhematics course taking. However, science course taking was not associated with school 
population groups, subsidized lunch participation, or school size. (See Figures C.19 through 
C.21 in Appendix C.) 

AVAILABILITY AND RELIABILITY OF DATA 

Are reliable, representative data available on a regular basis to describe course 
completion at the secondary level? 

There is only one recent nationally representative source of data regarding high school 
graduates' course completion patterns, the 1987 High School Transcript Study (Westat, 
1988). Unfortunately, the study is not ongoing, it does not contain information about the 
characteristics of the schools that students attended, and it does not provide a detailed 
tabulation of computer science courses. The first of these is the most serious limitation from 
the point of view of an indicator system. To our knowledge there are no current plans to 
undertake transcript studies on a regular basis in the future. (The other two problems could 
be remedied easily in any future national transcript studies.) We assume that high school 
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Fig. 4.6— Mathematics Courses Taken Tfaroui^ Grade 11 by School Type 



transcripts will be collected and analyzed by the National Education Longitudinal Study 
(NELS) in 1992, or shortly thereafter, when the original eighth grade cohort graduates from 
high school. These transcripts would provide a database from which comparable 1992 
course*completion indicators could be developed, but they will not address the long-term 
problem of ongoing transcript data. 

Because of the limitations of the 1987 transcript data, we supplemented the analysis 
with information from the 1985-1986 NAEP.^ NAEP is an ongoing data collection effort, so it 
overcomes the most serious problem with the transcript study. However, NAEP has 
shortcomings of its own. Most important, students were in the 11th grade at the time they 
participated in NAEP, so the database lacks any information about courses taken in 12th 
grade. In addition, there were some inconsistencies in the data regarding physics course 
taking. The percent of students saying they took physic$ was higher in high-minority schools 
than in low-minority schools, which nms counter to all other evidence about course 

^The High School Transcript Study was designed to use the same sample of schools and students 
as the 1985-1986 NAEP. However, operational problems interfered with this plan, and while the final 
school sample matched that in NAEP, the final student sampk did not. 
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Fig. 4.7— Science Courses Taken Through Grade 11 by Size and Tyx>e of Commimity 



completion among students in these schools. We suspect that students responded positively 
to questions about physics when they had actually taken physical science. 

Using the two data sources, it was possible to compare course-taking behavior through 
grade 11 with course-completion patterns of the same cohort of students after graduation. 
This comparison provided information about the reliability of the data. We would expect the 
percentage of students completing basic courses in mathematics and science to be 
approximately the same for the two groups (since these courses are usually taken prior to 
grade 11). In contrast, the percentage of 11th grade students completing advanced 
mathematics and science courses should be significantly lower than the percentage 
completing these courses a year later upon graduation. This is exactly what the data reveal. 

Looking toward the future, the major problem with NAEP has already been addressed; 
the survey will be administered to 12th grade students in 1990 and thereafter. However, 
there are still some problems that will have to be resolved if data from NAEP are to be used 
to estimate student course-completion patterns. First, NAEP course-taking data are derived 
from student self-reports rather than transcript analyses. Although previous research has 
found a high correlation between self-reported grades and course-completion information and 
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comparable data taken from transcripts (National Center for Education Statistics [NCES], 
1984; Valiga, 1986), the correlations were far from perfect. In one study the overall match 
between student reports of courses taken and transcripts was 95 percent; however, the 
percentage of errors was greater than 10 percent for certain mathematics and science courses 
(Valiga^ 1986). In another study, the correlations between student reports of coursework 
taken in mathematics and science and values obtained from transcripts ranged from .63 to 
.70 (NCES, 1984). The conclusion we draw from these results is that self-reported course- 
taking data are reasonably accurate, although transcript-based data are preferable. 

Second, NAEP surveys are administered in the winter and spring. Consequently, they 
cannot reflect course completion through the end of the school year. NAEP must either 
report course taking through the first semester or must rely on student projections of course 
taking for the remainder of the year. The reliability of such projections is unknown. In 
either case it would be appropriate to undertake separate benchmarking studies to validate 
the results of future NAEP assessments against actual student transcripts. Finally, as noted 
previously, NAEP needs to expand the range of courses on which it collects data and improve 
the quality of its data regarding school background characteristics. 
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EVALUATION OF INDICATORS 

Wfiat are the strengths and weaknesses of alternative indicators of course completion at 
the secondary level? Are they valid for the intended purposes and resistant to corruption? 

A variety of statistics have been used to summarize student course enrollment 
patterns. These include the percent of graduates completing specific courses or combinations 
of courses (Westat, 1988), the number of sections of each course offered per 100 students 
(Oakes et al., 1990), the distribution of credit hours completed through graduation (Tuma et 
al., 1989), the percent of coll^e-bound seniors (who took fJie SAT) reporting completion of 
specific courses (Educational Testing Service [ETS], 1989), and the percent of 11th grade 
students reporting completion of specific courses (Dossey et al., 1988; Mullis and Jenkins, 
1988; Goertz, 1989). Indicators could be developed using any of these statistics. 

We used three criteria for choosing among these alternatives when developing 
indicators: the indicators should (1) reflect the coiirse-completion behaviors of individual 
students, (2) differentiate between courses bv title or level (rather than assigning an equal 
value to all courses in a subject), and (3) include advanced college-preparatory courses, which 
usually are taken during the senior year. Consequently, the course-completion indicators we 
examined were almost all based on the percentage of high school graduates of a particular 
type (such as female students, Hispanic students, md students attending private schools) 
who completed a course or combination of courses. 

Many of the evaluative comments made in the previous discussion of course- 
availability indicators — the interpretability of an indicator based on course titles and the use 
of extreme groups and extreme group ratios — are relevant to course-completion indicators as 
well, and they will not be repeated here. However, there are three additional issues that 
relate to the definition and interpretation of cours^i-completion indicators. The first issue is 
whether to define indicators in terms of single coui'ses or combinations of courses. The 
second is the possibility of identifying early indica1x)rs that predict later course-taking 
behavior; the third is the problem of identifying a restricted set of courses to include in an 
indicator system. After discussing these topics, we will consider the corruptibility of course 
completion indicators. 

A careful review of the information contained in Tables 4.1 and 4.2 shows that the vast 
m^ority of students who completed the traditional college-preparatory courses in 
mathematics and science did so in the same combinations. For example, 45 percent of 1987 
high school graduates completed chemistry, and 43 percent completed chemistry and biology, 
thus only 2 percent completed chemistry but not biology. Similarly, only 3 percent completed 
physics, but not biology and chemistry. One implication of this might be that an indicator of 
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course completion based on single precollege mathematics and science courses would have 
provided roughly the same information as an indicator based on combinations of courses. 
However, a recent study of course taking in five districts in two states found marked 
differences in the order in which science courses were taken (McDonnell et al, 1990). 
Furthermore, at least one major science curriculum reform effort has called for chsjiges in 
the traditional grade levels at which core science courses are presented (American 
Association for the Advancement of Science [AAAS], 1989). As a result, indicators based on 
single courses may become less descriptive of student course completion behaviors in the 
fiiture, particularly in the fi#^ld of science. 

The second issue to explore iu the possibility of developing early indicators that predict 
advanced course-taking behavior. For example, it does not seem unreasonable to hypothesize 
that students who complete algebra I in eighth grade (ahead of the normal sequence) are 
more likely than other students to complete advanced or accelerated mathematics courses in 
high school. In the other direction, students who have not completed algebra I by the time 
they finish the ninth grade or biology I by the time they finish the 10th grade (behind the 
normal sequence) are less likely than other students to complete advanced or accelerated 
math and science courses. To the extent that these hypotheses are true, they permit the 
development of early predictors of the flow of students into the mathematics and science 
pipeline. 

Evidence from a nonrepresentative transcript study suggests that standardized 
course-by.grade sequence.s do exist in many districts, and that both prediction hypotheses are 
true to a certain extent (McDonnell et al.. Technical Report, in press). Students who took 
algebra I in grade eight or nine were two and a half times as likely to take geometry as 
students who took algebra I later. Alternatively, less than 10 percent of the students who 
took biology in a grade other than 10 took any other science courses. The authors concluded 
that lead indicators of the type described here were moderately usefiil in predicting 
mathematics and science course completion. Furthermore, the predictive power held for 
minority as well as majority students. 

Additional research needs to be done to test the validity of this type of course, 
completion prediction on a representative sample of students. However, initial evidence 
suggests that it would be possible to develop advanced indicators of the mathematics and 
science pipeline based on eighth and ninth grade course completion. 

Third, one of the problems that complicates the development of both course completion 
and course availability indicators is the large number of different courses offered by schools. 
Even after restricting the presentation to common mathematics, science, and computer 
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science courses, the tables in the previous chapter contained 35 different courses. This is too 
much information for efficient review. The tables fail to meet one important criterion of an 
effective indicator; they are not a parsimonious set of statistics that describe the status of 
mathematics and science curriculum. Furthermore, this encyclopedic approach displays 
course data without reference to the nature of the courses (e.g., level, difficulty, sequence, 
etc.). The reader must rely on his or her own understanding of curriculum to interpret the 

results meaningfully. 

There are a number of ways one might try to select subsets of courses or define 
clusters of courses to focus the presentation of course-related results (be they course 
completion or course availability). We will briefly consider two methods: courses clustered 
by level and subsets selected on the basis of relationships between course completion and 
other student outcomes or behaviors. A third approach, selecting key individual courses, was 
discussed previously. 

A natural way to cluster courses is by level: introductory, intermediate, and advanced. 
These clusters would be constructed by taking the typical college-preparatory sequence and 
subdividing it into groups of courses based on sequence and grade level. Introductory courses 
are those taken by college-preparatory students in grades 9 and 10, intermediate courses are 
taken by college-preparatory students in grades 10 and 11, and advanced courses are those 
taken by college-preparatory students in grade 12. In fact, the correspondence between 
sequence, grade level and difficulty is quite high, so it might be accurate to identify these as 
levels of difficulty. 

Using this designation, one can report that introductory college-preparatory 
mathematics courses (algebra I and geometry) are available in over 90 percent of high 
schools, while intermediate courses (such as algebra II and trigonometry) are available in 
approximately 60 percent to 90 percent of the schools. Already the problem becomes 
apparent. Although the cluster of intermediate course may make sense from the point of 
view of grade level, there is no sensible way to summarize statistics from the individual 
courses to obtain a clusterwide value. What is true of algebra II— in terms of course 
availability, course completion or any other feature— will not necessarily be true of 
trigonometry (or any other course). This suggests that the courses in the cluster are more 
dissimilar than similar. 

Alternatively, it should be possible to select a subset of courses that have particular 
significance and include only these in the indicator system. For example, recent evidence 
suggests that students who successfully complete algebra and geometry are more likely to be 
admitted to college and to complete college than students who do not complete this sequence 
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of courses (Pelavin and Kane, 1990). These results give special significance to the completion 
of algebra and geometry as predictors of college success^ and these courses have been referred 
to as gatekeeping courses. 

However, there are many other courses that represent important mathematics- or 
science-related curriculum milestones (e.g.^ a course that is needed for college admission, a 
course that seems to differentiate between vocational and college-bound students, or a course 
that identifies students who are likely to pursue advanced mathematical and scientific 
studies). The subset of courses that is relevant to one decision or one policy question will not 
necessarily provide information that is relevant to another. If one were interested in 
curriculum reform, it would be important to monitor courses with changing availability. If 
one were interested in preparing students to be admitted to college, it would be important to 
monitor the core of courses that corresponded to basic admissions requirements. If one were 
interested in the production of future mathematicians and scientists, it would be important 
to monitor the most advanced courses, those whose content was equivalent to college-level 
courses. There appears to be no convincing argument, either theoretical or empirical, for 
choosing a specific subset of courses to monitor. 

Another problem in identifying a subset of courses for inclusion in an indicator system 
is that selections may not retain their relevance over time. For example, a comparison of 
science courses offered in 1977 and 1988 showed rapid growth in the^availability of courses in 
Life Science, Earth Science, and Physical Science. (See Table 4.7.) Hc^wever, many people 
would have omitted these courses from a reduced indicator set deemed relevant in 1977, thus 
failing to reveal the only dramatic changes in science course availability that occurred during 
the next decade. Furthermore, as curriculum reform accelerates, it will become more 
difficult to select subsets of coiirses that will be of lasting relevance. For all these reasons we 
are less than sanguine about the possibility of selecting a reduced set of courses to use as 
indicators of course completion and course availability. 

Finally, it is important to consider how measures of course completion will behave in a 
highly charged indicator context. Fortunately, measures of course completion may be 
somewhat more robust than indicators of course availability. In general, students' course- 
taking behaviors are not easily manipulated, and there is little likelihood that an indicator 
system, in and of itself, would unduly influence student decisions. Were it easy to change 
students' choices, enrollment in advanced courses would have increased far more 
dramatically than it has. To the extent that national attention on course completion 
encourages qualified students to take math and science courses they might not otherwise 
elect to txike, it is probably to the good. 
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On the other hand, there is some concern that pressures to increase the percentage ( 
students taking more advanced courses will lead to inappropriate placements. This would 
detrimental to the students, who would be placed in courses for which they were not 
prepared, and to the curriculum, which might have to be changed to accommodate 
underprepared students. While we doubt that a national indicator system would have this 
strong an effect, there is some evidence that such changes could occur. Certainly caution 
would be warranted if local authorities were to adopt national standards as local 
accountabiUty measures. Supplemental research regarding the content of courses would t 
needed to provide baseline data to determine whether courses were being watered down d 
to the high visibility of course completion indicators or due to other causes. 
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5. COURSE CONTENT 



To this point the portrait of secondary mathematics and science curriculum has been 
drawn at a very high level of aggregation — ^years of study, courses available, and courses 
completed. While the results reported in the previous chapters provide some useful 
information about high school students' exposure to mathematics and science, none of the 
data directly describe the content of the classroom curriculum, e.g., what it taught, how it is 
tau^t, and what resources are used in teaching it. Even indicators based on specific course 
titles provide only indirect evidence of students' opportimity to learn specific knowledge and 
skills.^ 

In contrast, the present chapter focuses on the actual subject matter of classroom 
instruction, i.e., the mathematical and scientific knowledge, skills, procedures, and 
dispositions to which secondary students are exposed. The current status of secondary course 
content is described first, followed by a discussion of the availability and quality of course 
content data and an evaluation of alternative indicators of course content at the secondary 
level. To organize the presentation, course content is broken down into three components — 
topic coverage, curriculum-specific instructional strategies, and instructional materials and 
resources. 

Measures of topic coverage describe which facts, procedures, algorithms, etc. are 
presented. In comparison, measures of instructional strategies describe how these facts, 
procedures and algorithms are presented, i.e., the specific approach taken by the teacher 
when discussing mathematical and scientific knowledge and skills.^ Finally, measures of 
instructional equipment and materials describe what resources were available to support 
instruction and to what extent they were used in mathematics and science classes. 

Unfortxmately, the major importance of this chapter may be in what it fails to reveal 
about curriculum rather than what it reveals. As will be illustrated below, current 



^Recent studies of course content suggest that a tremendous amount of curriculum 
differentiation exists within courses and within grade levels (McDonnell et al., 1990; McKnight et al., 
1987), so course taking can be interpreted as a measure of exposure to subject matter only in very 
genex^ terms. 

Excluded from this discussion are measures of general instructional style (e.g., pacing, 
management techniques, rapport, etc.). 

n 
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nationally representative surveys provide little information about most aspects of course 
content in mathematics and science at the secondary level.^ 

CURRENT STATUS 

What is the content of secondary-level mathematics and science courses and how has 
this content changed over time? 

The principal theme that will be repeated throughout this chapter is the limited range 
of measures that exist to describe course content Neither topic coverage nor instructional 
strategies can be depicted in any detail, althou^ there are relatively more data available on 
mstructional equipment and materials. Some information will be presented regarding 
certain aspects of selected courses (primarily eighth grade general mathematics and science 
courses and precalculus/calculus courses), but the discussion will be limited. 

The following discussion has three major purposes: 



1) To provide information about the status of mathematics and science course 
content in a narrow range of courses; 

2) To illustrate the types of course content descriptions that would be possible on a 
broader scale if information were gathered more extensively; 

3) To highlight the gaps in our knowledge of course content that could be filled only 
through additional measures not presently available. 

Topic Coverage^ 

Three aspects of topic coverage will be described: which topics did teachers intend to 
present y which topics actually were presented, and what was the nature of the presentation, 
e.g., was it introductory, review of past material, or coverage in depth;'' did it receive major 

^Mo6t of the results are drawn from the Second Int<?mational Mathematics Study (SIMS), and 
they may be of limited generalizability because they are almost a decade old and because the U.S. 
samples were not representative of the nation. 

^e reader should be alerted to the fact that the view of "topic coverage* implicit in these 
results is not universally held. The approach to subdividing content taken by most surveys suggests 
that the substance of matiiematics and science can be divided into discrete pieces that can be measured 
and, by extension, tatight in isolation. Current research on scientific thinking suggests that 
mathomatics and science are highly integrated disciplines that are best learned in a real-world context. 
The notion of topic coverage outlined in the following paragraphs tends to ignore these complexities. 
(This concern is elaborated in the discussion of the validity of course content indicators later in this 
chapter.) 

'^We follow the lead of SIMS in interpreting information on coverage as a measure of curriculum 
sequence rather than instructional practice, i.e., what was the place of the particular topic in the 
developing sequence of mathematical content— was it review, taught in depth, or only alluded to as an 
introduction to future teaching? 
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exnphasis, minor emphasis, review only, or was it not covered at all? No data regarding 
trends in topic coverage were available. 

The discussion will include information about two groups of students, those in grade 8 
(regardless of course title) and those in precalculus or calculus classes (regardless of grade, 
thoiigh typically in grade 12). The former group comprised the population A sample used in 
SIMS as well as the initial cohort of ]?iELS; the latter group was the population B sample 
from SIMS. Because many people are unfamiliar with the eighth grade mathematics or 
science curriculum, comparative results may be more meaningful than absolute ones. In this 
case most of the data were drawn from SIMS, an international survey, so many of the results 
compare the situation in the United States to the situations in other countries. Results from 
SIMS were reported in Crosswhite et al. (1986). 

The SIMS survey attempted to differentiate the intended curriculum from the 
implemented curriculum. It included questions about the importance of topics within 
courses, the intended coverage of topics by teachers, the actual amount of classroom exposure 
to topic areas, and the nature of the exposure. For example, Table 5.1 shows mathematics 
educators' ratings of the importance placed on various topics in precalculus/calculus courses. 
There were noticeable differences between the rating in the United States and in other 
countries. For example, international mathematics educators placed greater emphasis than 
U.S. mathematics educators on elementary functions and calculus, probability and statistics, 
and finite mathematics. The U.S. sample was not adequate to estimate the degree of 
variation within the country. 

The intended curriculum also was measured by asking teachers to project the number 
of days they would spend on each major topic area during the year. Teachers in grade eight 



Table 5*1 



Topic Importance Ratings for U^. and 
International Calculus Curriculum 



Topic 



U.S. 



INT 



Sets and relations 
Number systems 
Algebra 
Geometry 

Elementary functions and 

calculus 
Probability and statistics 
Finite mathematics 
Computer science 
Logic 

Average 



5.0 
7.6 
8.3 

5.3 



6.2 
2.0 
4.0 
4.0 
4.0 



5.1 



3.8 
7.4 
7.3 
4.0 



8.6 
4.4 
6.0 
4.0 
0 



5.0 



SOURCE: CroMwhite et al» 1966. 
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intended to devote as much time to sets, relations, and properties as to algebra and geometry 
combined. Little or no attention was to be paid to probability or statistics, topics which have 
received greater emphasis in recent years in the college curriculum. Teachers of calculus 
classes in the U.S. SIMS sample intended to place a heavy emphasis on differentiation and 
integration with much less time spent on the related concepts from analytic geometry. 

Actual topic coverage did not always match intended coverage (though it is difficult to 
draw direct comparisons from the SIMS results because topics were classified in different 
ways for different questions). Although the vast majority of U.S. calculus classes covered the 
topic area that was rated most important internationally— elementary functions and 
calculus-/eu;er than half of the calculus classes in the U.S. sample covered seven of the eight 
curriculum topic areas. On the other hand, topic coverage in the United States was greatest 
for those topics rated the highest in importance by U.S. mathematics educators. 

Other international comparisons of eighth grade topic coverage also show U.S. classes 
lagging behind other countries. Eighth grade topic coverage in the United States was at or 
below the median for the 12 participating coiin tries and provinces in five of the six t4)pic 
areas (numbers and operations; relations, functions and algebra; geometry; measurement; 
data organization and interpretation; and logic and problem solving) measured in the 
International Assessment of Educational Progress (lAEP) (Lapointe, Mead, and Phillips, 
1989). 

In an attempt to explore the sequence of topics across courses, teachers participating 
in SIMS were asked to indicate whether topics were covered dxmng the current year, during 
previous years or not at all. These results differed markedly from those just reported. 
Teachers indicated that students in the majority of calculus classes in the U.S. SIMS sample 
had been exposed to seven of the eight calculus topics during the current year or during prior 
years. (See Table 5.2.) 

Another way one might measure exposure is to determine the number of class periods 
that were devoted to studying each topic and the nature of the exposure — studying the topic 
as new, review, or review and extend. Responses from calculus teachers to an abbreviated 
list of subtopics in analytic geometry are summarized in Table 5.3. The information in the 
table permits one to differentiate between subtopics that received a lot of attention overall 
(equations and graphs of conies) and newly introduced subtopics that received a lot of 
attention (vectors and vector operations). U.S. precalculus and calculus classes in the SIMS 
sample spent relatively little time reviewing topics introduced previously in comparison with 
the time that was spent reviewing and extending topics or introducing new topics. It should 
be noted that these patterns may not be typical of all U.S. mathematics courses; calculus 
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Content Exposure by Topic 



Percentage of U.S. Classes 







Precalculus 






Calculus 






Taught 


Taught 


Never 


Taught 


Taught 


Never 


Topic Area 


Before 


This Year 


Taught 


Before 


This Year 


Taught 


Sets/relfiitions 


31 


50 


19 


50 


40 


10 


Number systems 


39 


42 


19 


75 


14 


11 


Algebra 


34 


52 


14 


53 


41 


6 


Geometry 


21 


40 


39 


41 


26 


3S 


Elementary functions and 


8 


37 


55 


9 


83 


8 


calculus 














Probability and statistics 


29 


14 


57 


50 


6 


44 


Finite mathematics 


29 


21 


50 


62 


8 


30 


Other areas 


8 


8 


84 


15 


4 


81 


Total 


22 


40 


38 


38 


45 


17 



SOURCE: Crouwhite et aL, 1986. 



Table 5^ 

Analytic Geometry Subtopic Coverage in Calculus Classes 
CAbbreviated) 





Average Number of Class Periods 






Review and 


Review 


Subtopic 


New 


Extend 


Only 


Parametric equation of line 


1.5 


.1 


.1 


Analjrtic proof of theorems 


1.3 


.2 


.2 


Equations and graphs of conies 


2.2 


3.6 


.5 


Vectors and vector operations 


2.9 


5L 


.1 


Translations of axes 


1.0 


.3 


.2 


Rotation of axes 


0.9 




.1 



SOURCE: CitMBwhite et al., 1986. 



courses attract the best students, and the balance between review and new material may be 
('ifferent than it is in courses with different groups of students. 

There are other ways one might try to measure curriculum emphasis. Eighth grade 
science and mathematics teachers who participated in NELS were asked to describe the 
emphasis they placed on a number of mathematics and science topics as one of the following: 
major, minor, review only, or not covered at all. In mathematics, algebra, problem solving, 
ratios and percents, and integers were major topics in the majority of the classes; probability 
and statistics, measurement, and geometry were major topics in less than one-quarter of the 
classes. In science, only earth science was described as a major topic in the majority of the 
classes. The next most frequently taught topics were weather/astronomy, chemistry, atomic 
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theory, and physics subjects (Horn and Hafoer, in press).® It is interesting to note that earth 
sciences and weather and astronomy were the most popular subjects, yet, according to 
NSSME, few junior high schools had access to telescopes (29 percent), weather stations (10 
percent), or portable planetariums (7 percent) (Weiss, 1987). 

Curriculum-Specific Instructional Strategies 

Classroom-level instructional strategies are an important element of curriculum, since 
the nature of the teacher's presentation determines students' learning opportunities. A topic 
can be presented in many different ways with equal enthusiasm and skill. For example, 
Boyle's Law (describing the relationship between temperature, pressure, and the volume of a 
gas) can be taught with an emphasis on the computational algorithm or on the conceptual 
relationships among the variables. A measure of topic coverage alone would not fully 
describe students' opportunity to learn in these two situations; additional measures of 
instructional approach would be needed to portray the differences accurately. 

Selected results from SIMS, NELS, and NSSME will illustrate some measures of 
classroom-level currictdum implementation that have been collected. This discussion 
presents information about the choice of goals and objectives for mathematics instruction, 
the representation of specific concepts, and the application of concepts. 

Table 5.4 shows the percentage of junior high school mathematics classes placing 
heavy emphasis on various objectives for mathematics instruction. It is interesting to note 
that the primary objectives for mathematics instruction changed little across grade levels; 
approximately the same percentage of elementary school, jiinior high school, and high school 
teachers emphasized the three most common objectives shown in Table 5.4. Three other 
goals were emphasized relatively more strongly in the lower grades: performing 
computations with speed and accuracy, becoming aware of the importance of mathematics in 
everyday life, and becoming interested in mathematics. The only goal that received 
substantially more emphasis at the secondary level was learning about the career relevance 
of mathematics (Weiss, 1987). 

Teachers also exercise choices about the ways in which concepts are represented or 
interpreted. For example, there are a number of ways to model the concept of a fraction. 
More than 80 percent of the teachers in grade eight emphasized the interpretation of 

^It is difBcult to know how to interpret these ratings, since no operational definitions were given 
for the levels of emphasis. This is an instance in which additional research would be necessary to 
validate any interpretation of the results. 
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Table 6.4 

Objectives of Mathematics Instruction 





Percent of Mathematics Clsisses 


Objective 


with Heavy Emphasis* 


Know mathematical facts, principles, algorithms, or procedures 


80 


Develop a systematic approach to solving problems 


76 


Prepare for further study in mathematics 


67 


Perform ccmiputations with speed and accuracy 


59 


Become aware of the importance of mathematics in daily life 


61 


Develop inquiry skills 


50 


Learn to effectively communicate ideas in mathematics 


54 


Become interested in mathematics 


40 


Learn about the applications of mathematics in technology 


27 


Learn about the career relevance of mathematics 


28 


Learn about the history of mathematics 


5 



teachers were given a 6-point scale for each objective, with 1 labeled "none," 2 "miuimal emphasis/ 4 
"moderate emphasis,' and 6 "very heavy emphaaia.* These nambera represent the total circling either 5 or 6. 



SOURCE: WeiM, 1987. 

fractions as decimals and as quotients, while fewer than 40 percent emphasized fractions as 
parts of regions, ratios, or points on a number line. 

Mathematical and scientific concepts have many applications in the real world, and 
teachers incorporate different applications into their lessons. For example, calculus teachers 
asked students to apply the concepts they learned in trigonometry to a variety of different 
situations. The most common were inaccessible distance problems (66 percent of teachers), 
other surveying problems (58 percent), periodic motion (42 percent), and vector applications 
(39 percent). Most problems were drawn from the textbook, although approximately 40 
percent of the teachers created their own problems. 

These examples illustrate the kind of information about instructional strategies that 
would contribute to a fuller description of mathematics and science curriculimi if they were 
collected more extensively. 

instnictional Equipment and Materials 

One factor that affects both the content and the quality of mathematics and science 
instruction is access to appropriate instructional equipment and materials. By instructional 
equipment and materials we mean textbooks, supplemental reading materials, instructional 
films and videos, calculators, computers, laboratory facilities, laboratory equipment, and any 
other tangible resources used to support the instructional program. 

Data will be presented separately to describe the status of textbooks, laboratory 
facilities, computers, calculators, and other equipment. Each picture was pieced togeUier 
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with information from many different, and incomplete, sources. As a result, the description 
is imeven, and it is rarely possible to describe trends over time. 

Textbooks were used in more than 90 percent of all mathematics and science classes, 
and this percentage remained essentially unchanged between 1977 and 1986 (Weiss, 1987). 
Because mathematics teachers make only limited use of other curriculum resources when 
setting content or teaching methods (McKnight et al., 1987), a content analysis of these 
textbooks might be used to reveal the common elements of the text-based curriculum 
underlying most mathematics classes. However, such an analysis would not necessarily 
provide an accurate picture of the actual instructional content of classes, since teachers 
tended not to cover all the material in their textbooks. Table 5.5 shows that almost one-half 
of the science teachers and almost one-third of the mathematics teachers covered less than 
three-quarters of the material in their textbooks. 

How good are the textbooks? Only indirect measures exist. For example, the majority 
of high school mathematics and science teachers did not consider textbook quality to pose>a 
serious problem for their instructional program. On the other hand, more than half of the 
science teachers were dissatisfied with the provision of examples to reinforce concepts and 
the quality of supplementary materials that accompanied the texts, and approximately two- 
thirds of the mathematics teachers were dissatisfied with the examples of applications of 
mathematics and with suggestions for calculator and computer i^e. Finally, approximately 
30 percent of the high school science classes were using textbooks published before 1980 
(Weiss, 1987). There are other methods for assessing textbook quality, such as expert 
reviews of pedagogical approaches and content, but these have not been done on a large 
scale. 

Science laboratories (including lab facilities in regular classrooms, general purpose 
labs, and specialized labs) were found in more than 70 percent of all high schools. (See Table 

Tables^ 
Textbook Coverage 



Percent of High School 
(Grades 10 to 12) Classes 



Percenter Textbook Covered 


Mathematics 


Science 


Less than 25 


2 


1 


25 to 49 


6 


11 


50 to 74 


24 


37 


75 to 90 


45 


34 


More than 90 


23 


16 


Unknown 


1 


1 



SOURCE: WeiM, 1987. 
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5.6.) Because many advanced science courses emphasize experimentation, the presence of 
laboratory facilities is particularly important. In fact, Figure 5.1 shows a direct relationship 
between the presence of specialized laboratories and the offering of Advanced Placement 
courses in science. Schools without specialized science laboratories are half as likely to offer 
AP biology and less than one-quarter as likely to offer AP chemistry or physics than schools 
with specialized science labs. In comparison, only 21 percent of schools reported that they 
had a mathematics laboratory (Weiss, 1987). 



Table 5.6 
Science Laboratory Facilities 



Type of Facility 


Percent of Schools 


Lab facilities in classroom 


75 


General purpose labs 


71 


Specialized labe 


70 



SOURCE: RAND tabulations of data from the 
1985-1986 National Assessment of Educational 
Progress. 



No specialized science lab 



Specialized science lab 




Biology Chemistry 

Advance Placement courses 



Physics 



SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 



Fig* S*!*— Advanced Placement Science Course Availability by Presence of Specialized 

Sci«Dc« LabaratoriM 
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Computer terminals and/or computers were much more widely available in 1986 than 
in 1977. Almost all high schools (98 percent) reported that they had computers or computer 
terminals available in 1985-1986 compared to only 36 percent of high schools in 1977 (Weiss, 
1987). Furthermore, the percentage of 17-year-old students reporting that they had access to 
a computer for learning mathematics more than doubled between 1978 and 1986 (Dossey et 
al., 1988). Similar results have been reported by other surveys conducted at approximately 
the same time (Quality Education Data [QED], 1985; Johns Hopkins University, 1986). All 
agree that the decade of the 1980s was marked by dramatic gru^vth in the number of 
computers in schools. While the annual rate of growth had declined by 1989, the absolute 
number of computers acquired each year continued at a fairly uniform rate throughout the 
decade (Becker, 1990). 

The fact that almost all schools have computers available may be misleading in a 
number of ways. It says nothing about the degree to which teachers and students can gain 
access to the computers, the degree to which computers are actually used by teachers and 
students, nor the purposes to which the computers are put when they are used. 

It is possible to construct a minipatchwork from a number of different sources to 
describe computer access and use more completely. The median number of computers per 
high school was 45 in 1989 compared to 21 in 1985 (Becker, 1990). Furthermore, the 
percentage of all schools with a minimum of 15 computers, "enough computers so that if they 
were located in one place, one class of students — working in pairs — could be 
simultaneously served,** doubled from 24 percent in 1985 to 57 percent in 1989 (Becker, 
1990). However, despite this dramatic growth, the number of students per computer was 
still quite high. In 1985 there was only one computer for every 31 students (on average) at 
the high school level (Johns Hopkins University, 1986). 

Ultimately the number of computers is not as revealing as the degree to which 
teachers and students can use computers. Table 5.7 shows that only 29 percent of 
mathematics teachers and 20 percent of science teachers reported that computers were 
"readily available" for instructional uses. Computer coordinators estimated that less than 
one*quarter of all 11th grade students used a computer for 30 minutes or more each week in 
any classes, and they estimated that only 8 percent of students' time on computers was spent 
learning mathematics and only 5 percent learning science^ (Martinez and Mead, 1988). 

Students, too, reported vei7 little computer use in mathematics and science. Only 29 
percent of 11th grade students reported that they had ever used a computer in mathematics 

''In contrast, at least one-half of all student time on computers was occupied learning computer- 
specific skills, e.g., how to use word processing and other software (Becker, 1990). 
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Table 5.7 

Availability of Computers for Science and Mathematics Instruction 



Percent of Classes 





Mathematics 


Science 


Not available 


24 


24 


Available but quite difficult to access 


23 


28 


Available but somewhat difficult to access 


23 


27 


Readily available 


29 


20 


Missing/inconsistent 


2 


2 


Unknown 


1 


1 



SOURCE: Weiss. 1987. 



class, and only 22 percent reported that they currently used a computer to practice 
mathematics. Only 15 percent of 11th grade students reported they had ever used a 
computer in science classes and only 13 percent had used computers to do science problems. 
Granted, computer use has increased. In 1985, only 11 percent of high school teachers 
reported that they used computers with their students during a typical week (Johns Hopkins 
University, 1986); by 1989, 41 percent of high school math teachers and 36 percent of high 
school science teachers reported that computers were used by students in at least one of their 
classes (Becker, 1990). However, much of this increase in use was only marginal. Only 23 
percent of math teachers and 11 percent of science teachers had students in any classes use 
computers to a "substantial" degree (Becker, 1990). 

With the exception of hand-held calculators, access to other types of equipment and 
materials either remained approximately the same or declined between 1977 and 1986. (See 
Table 5.8.) However, although the availability of calculators at the school level is high, 
student use is still quite low. Although 94 percent of high school principals reported their 
school had hand-held calculators available for use (Weiss, 1987), only 26 percent of grade 11 
students reported their schools had calculators for use in mathematics classes (Dossey et al., 
1988). At the eighth grade level, 60 percent of the students had some access to calculators in 
mathematics classes, but only about one-third used the calculators more than once each week 
(Horn and Hafner, in press). 

Another way to examine the use of instructional equipment and materials is to 
investigate the tise of specific tools in specific subjects. For example, 82 percent of the 
students in calculus classes used calculators for solving trigonometric problems, 73 percent 
used trigonometric tables, 15 percent used logarithms and trigonometric tables, and 12 
percent used computers. In contrast, calculus teachers used few instructional aids. Only 19 
percent used surveying instruments, circle function plotters or graphing devices (Crosswhite 
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Table 5^ 

Availability of Selected Instructional Equipment and 
Materiala 



Percent of High School 
Classes (10 to 12) 



Equipment 1977 1985-1986 



Greenhouse 


26 


30 


Telescope 


29 


29 


Darkroom 


75 


73 


Weather station 


22 


7 


Hand-held calculators 


77 


94 


Microscopes 


95 


99 


Cameras 


81 


84 


Models (e.g., the solar system) 


79 


82 


Small group meeting room 


59 


59 


Learning resource center 




67 


Outdoor study area 




31 


Vivarium 




3 


Portable planetarium 




6 


Videocassette recorder 




98 


Videodisc players* 




43 



*The data on availability of videodisc players are suspect; they are 
much larger than those reported by other recent studies. It is possible 
that some principals did not distinguish between videodisc players and 
videocassette players. 



SOURCL: Weiss, 1987. 

et al., 1986). One can assume that many of the teachers plot functions on the chalkboard 
without aids, but it is still interesting to note how few aids they use. 

How evenly is course content distributed cLcross secondary schools and students? 

Little information is available regarding the differentiation of topic coverage by school 
or student characteristics. The U.S. SIMS sample was too small to permit subsample 
comparisons. Results from NELS, which do permit some such comparisons, are just 
beginning to appear. The following examples illustrate the type of analys^'tii^t would be 
possible were such data more widely available. Findings regarding NELS were drawm from 
Horn and Hafher (in press). 

Topic Coverage 

The U.S. mathematics curriculum in grade eight was much more diffuse than tlie 
Japanese, Belgian, or French cuiricula, placing intermediate emphasis on a larger number of 
topics. The U.S. curriculum lacked "intensity" or sustained focus on any ^ ingle aspect of 
mathematics (McKnight et al., 1987), For example, che Japanese curriculum in the seventh 
grade provided an intensive introduction to algebra with a strong secondary emphasis on 
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geometry. The Belgian and French cxuricula placed primary emphasis on geometry with 
considerable attention to fractions. (In a similar manner, regional, state, or district 
comparisons could be made within the United States if the sample had been drawn with this 
purpose in mind.) 

In the eighth grade, exposure to topics was associated with class level, school type, and 
certain school and community characteristics. Algebra I classes in grade eight covered more 
algebra and enriched topics than pre-algebra or general math classes (Horn and Hafher, in 
press). However, content coverage varied considerably across classes. In fact, some pre- 
algebra and general math classes exposed students to as many adgebra and enriched math 
topics as some algebra I classes (McDonnell et al., 1990). 

Eighth grade students in private schools were more likely to be exposed to algebra 
topics than eighth grade students in public schools. The most common topics taught in 
public schools were ratios and percents, problem solving, and fractions, while the most 
common topics in private schools were algebra, problem solving, and integers. 

Similarly, eighth grade students in suburban schools were more likely to be exposed to 
algebra as a major topic than were students in urban and rural schools. Within public 
schools, the percentage of eighth grade students for whom algebra was a major topic was 
inversely related io the percentage of students in the school who were receiving subsidized 
lunch. 

Curriculum-Specific Instructional Strategies 

The following results illustrate the types of distributional analyses that would provide 
usefiil information about curriculum implementation. 

Secondary teachers' emphasis on instructional objectives were affected far more by 
student abilities and class groups than by school characteristics. For example, teachers of 
high-ability classes were significantly less likely to emphasize the importance of science and 
mathematics in everyday life and the importance of computations than were teachers in low- 
ability classes. Instead, teachers of high-ability classes placed more emphasis on objectives 
such as inquiry skills, lab techniques, interest, basic concepts, and facts and principles 
(Oakes etal., 1990). 

On the other hand, some instructional variables were related to school and/or student 
characteristics, including the amount of time spent in whole group instruction, the frequency 
of science experiments, and the amount of homework assigned. For example, students were 
more likely to receive whole group instruction in mathematics and science if they were in 
Catholic schools than public or private schools; 70 percent of Catholic school students 
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compared to 49 percent of public school students attended mathematics classes in which 50 
to 75 percent of the time was spent in whole groups, and similar results were reported for 
science instruction. 

The amount of time spent in whole group instruction also was related to subsidized 
lunch participation, race/ethnicity, and school environment variables, but the relationships 
were different for instruction in mathematics and science. Students in schools with high 
subsidized lunch participation spent more time in whole group instruction in mathematics 
than did students in schools with low subsidized lunch participation, but the latter group 
spent mor<i time in whole group science instruction than the former. The implication seems 
to be that small group instruction in mathematics is often remedial, whereas small group 
instruction in science involves participation in science experiments. 

Private school eighth grade science teachers were more likely to conduct weekly 
science experiments and assign homework than were other teachers. Private school science 
classes were also smaller than classes in other schools. Specifically, 66 percent of private, 
nonreligioias school science teachers conducted weekly science experiments compared to 55 
percent of teachers in Catholic schools, 47 percent of teachers in public schools, and 9 percent 
of teachers in private religious (non-Catholic) schools. The pattern was similar for class size: 
58 percent of students in private religious schools were in classes with no more than 15 
pupils compared to 41 percent of students in private nonreligious schools, 14 percent of 
students in Catholic schools, and 11 percent of students in public schools. 

Instructional Equipment and Materiais 

Available data permit only incomplete descriptions of the distribution of textbooks, 
laboratory facilities, computers and calculators. 

Science laboratory facilities, particularly specialized science labs, were not xiniformly 
distributed across schools. Larger schools were much more likely than smaller schools to 
have laboratory facilities of any type. (See Figure 5.2.) In addition, hig-metropolitan schools 
were more likely to have general and specialized labs, while extremely rural schools were 
more likely to rely on laboratory facilities in class. (See Figure 5.3.) There was only a mild 
relationship between school population groups and subsidized lunch participation and the 
availability of laboratory facilities. 

Computers were less available to teachers and stu^^ents in schools with a low percent 
of parents in professional occupations and inner-city high schools than in high schools with a 
high percent of parents in professional occupations. Although 95 percent of the principals in 
high occupational-profile high schools reported that they had access to computers for 
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SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 

Fig. 5J2 — ^Presence of Science Laboratories by Grade Level Enrollment 



instructional use, only 77 percent of the principals of low occupational-profile high schools 
reported availabiUty of computers for instruction. Similarly, teachers in low occupational- 
profile and inner^city schools reported that computers were less readily available at their 
schools, and students in high-minority, inner-'^ity schools had access to far fewer computers, 
even when they were available. (Oakes et aL, 1990). 

Although Oakes et al. reported there were fewer computers available in high-minority 
schools, almost the same percentage of minority and nonminority 11th graders reported they 
had ever used a computer, and approximately the same percentage of the three groups were 
actually studying computers. ^'See Table 5.9.) However, almost 50 percent more white 
students than black or Hispanic students reported that their families owned a computer. 

Telescopes, calculators, and other materials were not evenly distributed across schools 
or students. Schools with high concentration of low-income or minority students and schools 
concentrated in the inner city had fewer calculators and other equipment available than did 
other schools. Furthermore, teachers in high-poverty, high-minority inner city schools 
reported greater problems associated with the lack of instructional resources than did 
teachers in other schools (Oakes et al., 1990). 
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Fig. 5.3 — Presence of Science Laboratories by Size and Type of Community 



Table 5.9 

Student Experience witb Computers 



Group 



Race/ethnidty 
White 
Black 
Hispanic 

School type 
Public 
Nonpublic 

Size and type of community 
High metropolitan 
Low metropolitan 
Extreme rural 



Percent of 1 1th Graders 



Used a 
Computer 



Studying 
Computers 



89 
81 
80 

87 
88 

93 
80 
90 



SOURCE: Martinez and Mead, 1988. 



20 
21 
19 

20 
22 

21 
22 
29 



Family Owns 
Computers 



32 
22 
21 

29 
44 

41 
22 
19 
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AVAILABILITY AND RELIABILITY OF DATA 

Are reliable, representative data available on a regular basis to describe course content? 

Despite the potential value of coiarse work infonnation for educators and policjonakers 
(McDonnell et al., 1990), such information has not received high priority in the design of 
ongoing national data collection efforts. 

For example, the most extensive information regarding the topics covered in the 
secondary mathematics and science curriculum in the United States comes from the second 
round of international mathematics and science studies, SIMS and SISS (the Second lEA 
Science Study), conducted in 1981-1982. Designed and conducted primarily by mathematics 
and science educators, these studies (and their predecessors in the 1960s) gathered 
information about the general representation of content areas in the U.S. curriculum and the 
opportunity provided to students to learn specific topics. Unfortunately, neither study 
assessed a representative sample of students (the SIMS sample is probably the more 
representative of the two^ so the results cannot be generalized with confidence, and the 
sample sizes were too small to permit any analyses by population subgroups. Furthermore, 
both studies are now almost a decade old, so the information is useful only as baseline data 
in an indicator system. 

The only national study that currently collects topic coverage data is NELS, a 
longitudinal study of 1988 eighth grade students. The limited topic coverage questions in the 
base NELS survey were expanded in the first follow-up to NELS administered in 1990-1991 
when the student cohort was in grade ten. There is every reason to believe that similar 
questions will be asked of the mathematics and science teachers responsible for the courses 
taken by the students when they are in grade 12 in 1992. However, NELS will continue to 
follow the 1988 eighth grade cohort as it progresses through school; it will not continue to 
supply data about the curriculum in the grades they have completed In addition, the 1988 
International Assessment of Educational Progress (lAEP) collected some data regarding 
students' opportunity to learn topics included in the test battery. However, the focus of this 
study was international comparisons of achievement, and only limited data were gathered 
regarding curriculum coverage within the United States. 

Similarly, no ongoing, representative source of information exists concerning 
curriculum-specific instructional strategies. The most extensive examination of these 
elements of instruction was conducted by SIMS in 1981. NELS included adaptations of items 
from SIMS in 1988 and 1990, and we anticipate they will continue to ask teachers about 
their implementation of curriculum in mathematics and science in 1992. However, that will 
probably be the last school-based survey from NELS, NSSME included a few questions 
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about teachers' attitudes toward instruction, which were reanalyzed by Oakes et al. (1990) to 
examine differential effects across schools and students. Occasionally, questions about 
instructional strategies appear in NAEP, but they are not standardized from assessment to 
assessment. 

Finally, although SIMS, NSSME, NAEP, and NELS each have examined certain 
aspects of instructional equipment and materials (e.g., calculator use, computer availability, 
satisfaction with textbooks), these have been isolated investigations. None of these studies 
had instructional resources as a major focus of attention; none had a framework for 
measuring the topic comprehensively. Furthermore, of the four sources of data used in this 
report, only NAEP is ongoing. 

Although the surveys fail to meet our needs in terms of coverage, replication, and 
sampling, they appear to satisfy our criterion for reliability. In the few instances in which 
results can be compared within or across surveys, they were reasonably consistent. 

EVALUATION OF INDICATORS 

What are the strengths and weaknesses of alternative indicators of secondary-level 
course content in mathematics and science? Are they valid for the intended purposes and 
resistant to corruption ? 

Because of the paucity of data regarding course content and the lack of a framework 
for analyzing and presenting the information, no specific course content indicators are 
recommended. Instead, we suggest that two or three courses, grade levels, or course/grade 
level combinations be selected as targets for further indicator-defining research. Because of 
the extensive amount of research already conducted at the eighth grade level, this is a 
natural starting place. However, much of this work fails to distinguish between courses 
within grade level, so we suggest additional targets be defined in terms of specific courses. 
One strategy would be to select three courses each in mathematics and science— one 
introductory, one intermediate, and one advanced — and try to develop a core of content 
measures for each course. Likely candidates would be algebra I, algebra II, and calculus in 
mathematics, and biology I, chemistry I, and physics I in science. 

It is considerably more difficult to decide which aspects of course content should be 
measured. Further research needs to be done within each subject area to evaluate 
alternatives, operationalize and validate measures, and define specific indicators. Ideally 
one would like to be able to measure breadth of coverage within a subject as well as depth of 
coverage in selected topic or thematic units. Instructional strategies and resources should be 
selected on the basis of the topic/tliematic units being assessed, and, to the extent possible, 
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they should be comparable across coiirses. Developing such measures will require 
considerable additional study, both to select useful concepts to measure and to develop 
effective measurement techniques. For example, teacher self-reports, which are an efScient 
way to gather such information, may fail the test of robustness in a high-stakes context. 

The following discussion will highlight some of the problems that are encountered in 
developing measures of topic coverage, instructional strategies, and instructional equipment 
and materials. 

Topic Coverage 

Information about topic coverage is critical to a mathematics and science indicator 
system. Measures of course availability and course completion assume a commonality of 
content within courses that may no .jcist. Measures of topic coverage are needed to help 
interpret the more highly aggregated measures based on course titles. In addition, topic 
coverage is of interest in its own right, because it illuminates the actual content presented to 
students and differentiates curriculum in a much more refined manner than indicators 
defined at the course level. However, there are a number of problems that must be overcome 
in order to develop indicators of topic coverage. 

The first difficulty encountered when tr3ring to measure topic coverage is the problem 
of definition. There are no guidelines regarding what constitutes a topic or a subtopic in 
mathematics or science, or how finely differentiated topics or subtopics should be. Individual 
discretion is a large factor in subdividing the curriculum into chunks of related content, and 
so the definition of topics is discretionary as well 

The clustering of mathematical or scientific facts and theories into groups (whether 
they are called topics or themes or ideas) is designed to maximize the similarity of the 
content within a group while maximizing the differences in content between groups. In 
theory, this could be done in innumerable ways. In practice, curriculum experts in 
mathematics and science have been able to agree that certain content distinctions are 
reasonable. In both international surveys cited here, educators were able to reach consensus 
on the definition of topics within secondary mathematics and science. Nevertheless, there 
may be theoretical and practical disagreement regarding the definition of topic categories 
used to subdivide curriculum. 

A related concern has to do with the level of detail with which topics are specified. 
Most comprehensive descriptions of the secondary mathematics or science curriculum include 
an exceptionally large number of topics and subtopics. For example, the SIMS content 
matrix for grade 12 precalculus and calculus included 19 topics and 150 subtopics 
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(Crosswhite et al., 1986). This is not necessarily a bad thing; it suggests the richness of the 
subject matter. However, an unfortunate consequence of the multiplicity of topics within a 
course is the difficulty of monitoring them all. Any practical topic coverage indicator would 
have to focus at a fairly coarse level of aggregation or on a subset of topics defined at a finer 
level. One negative consequence of the latter approach is the tendency to trivialize the 
curriculum. 

Certainly, the goals of the curriculum should be reflected in tlie selection of topics to be 
monitored. This may not be as simple as it seems, because different policymakers emphasize 
different curriculum goals. For example, it may serve NSFs purpose to focus on topics from 
advanced mathematics and science because these courses prepare the students who are 
likely to become the mathematicians and scientists of the future. However, others might 
focus on different goals. For example, the National Research Council (1989) notes that 
"students rarely learn mathematics appropriate to enlightened citizenship or to the needs of 
the workplace.** A topic coverage indicator focused on mathematics for enlightened 
citizenship would reflect different topics than one targeted at future mathematicians and 
scientists. 

Assuming that these problems can be resolved, there is a more fundamental problem 
with topic coverage information that must be considered before such indicators are 
developed. The tendency to try to break the curriculum into discrete pieces runs counter to 
current understanding about the process of learning mathematics and science. Learning 
mathematics and science is no longer seen as the mastery of isolated pieces of information 
but as a process of constructing meaning in context. The fractionalization of content that 
characterizes attempts to assess topic coverage is diametrically opposed to curriculum reform 
efforts, which aim to integrate mathematical and scientific thinking in the service of real 
world questions and problems (National Council of Teachers of Mathematics [NCTM], 1989). 
Topics, as commonly defined, fail to reflect current beliefs about the integrated structure of 
mathematics and science. A more complete description of mathematics and science would 
measure something about the topics discussed and the nature of the cognitive learning that 
takes place. 

For example, it might be possible to collect data that portrayed mathematics and 
science in more modem terms by basing the data collection on a different type of curriculum 
taxonomy. The mathematics curriculum and evaluation standards (NCTM) might be used to 
develop measures in a manner more congruent with the principles of organization that guide 
current curriculum reform. The major thematic units might be problem solving, 
communication, reasoning, mathematical connections, etc., rather than ratio and proportion, 
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sets and relations, etc. In a, similar manner, one might develop a taxonomy based on the 
nature of science, mathematics and technology or on the basic knowledge of the world from 
the perspective of Uiese disciplines (American Association for the Advancement of Science 
[AAASl, 1989). 

Finally, one must be careful when developing topic coverage indicators not to 
emphasize breadth of coverage at the expense of depth. Whichever approach one adopts to 
classifying course content, it is important to learn more than merely whether or not a topic 
was presented. Par more useful is knowledge of the amount of time spent discussing the 
topic, the importance placet} upon it, the degree to which it is elaborated, the other topics to 
which it is related, etc. SIMS, NELS, and SRA (the School Reform Assessment project) 
provide models of alternative methods for exploring some aspects of the depth of coverage of 
a particular topic. Supplemental research is needed to refine topic coverage indicators. In 
particular, research to assess the consistency of topic coverage within course titles would 
help make both types of indicators more meaningful. ^ 

Instructional Strategies 

The instructional strategies employed by teachers affect students' opportunity to learn 
mathematics and science as much as any other element of the curriculum. Consequently, 
measures of classroom-level curriculum implementation strategies would be desirable 
elements in an indicator system. Yet these aspects of curriculum have not been measured 
systematically; they remain largely unexamined and unreported. 

Curriculum-specific instructional variables include such items as: 



(joals for learning 
Content organization 
Mode of presentation 
Classroom organization 
Representation of concepts 
Expectations for students 
Homework policies 



The potential impact of these factors can be seen by translating them into classroom 
terms. For example, the opportunity to learn mathematics and science will differ 
considerably if teachers have different goals (one striving to increase students' ^<icility with 
algorithms, another emphasizing the relationship of topics to other subject fields); different 
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content organization (one basing the order of presentation on the order of historical 
development, another on the relationship of topics within real world problem situations); 
different modes of presentation (one stressing lecture and written problems, another 
experiments and hands-on explorations); different classroom organization (one preferring 
whole class activities, another small group work); different representations for concepts (one 
^:hoosing a geometric model to illustrate a concept, another an algebraic model); different 
perceptions and expectations for students (one expecting students to master the material by 
the end of the course, another hoping only to introduce the concepts); and so on. The choice 
of approach will affect both the skills and concepts students have an opportunity to learn and 
the ways they have of learning them, i.e., it will affect the curriculum. A portrayal of 
curriculum absent information about these aspects of classroom implementation clearly is 
incomplete. 

One major impediment to expanding the scope of instructional practice measures 
collected in national surveys is the lack of a conceptual framework for structuring the effort. 
Another is the paucity of empirical work regarding the definition and validation of indicators 
of curriculum*specific instructional constructs. Much research needs to be done to 
understand how best to define and operationalize constructs within the domain of 
curriculum-specific instructional strategies. Supplemental research to design and validate 
practice measures (such as McDonnell et al., 1990) are necessary first steps. 

Furthermore, we need to understand better the relationship between instructional 
strategies and outcomes to help determine which instructional features are most important 
to monitor. Indicators of curriculum-specific instructional strategies should at least be 
sensitive to research on effective teaching techniques, such as small group cooperative 
learning, problem posing, writing and communication, etc. (Cooney and Hirsch, 1990). 
Finally, priorities for developing indicators of instructional variables should depend on the 
goals that are set for mathematics and science education, because different indicators will be 
relevant to different instructional goals (Oakes and Carey, 1989). 

The measurement problems may be greater in this area than in the other areas of 
course content because of the need to rely on teacher self-reports. A concern has been raised 
about the tendency of teachers to give socially desirable answers when asked about specific 
practices within their classrooms (McDonnell et al., 1990). There is no basis forjudging the 
effects of social desirability on the results of previous studies, but it should be a concern in 
the development of classroom content measures in the future. Supplemental research to 
assess the validity of responses obtained from classroom teachers should be an important 
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part of any effort to incorporate teacher-based measures of course content into a national 
indicator system because the stakes associated with performance may be quite high. 

Instructional Materials and Equipment 

Although instructional equipment and materials may not appear to be as significant 
an element of curriculum as some of the school- and course-level constructs presented 
previously, the development of indicators of instructional equipment and materials can be 
justified on a number of grounds. First, certain resources, such as textbooks, play an 
extensive and important role in instruction and must be considered core elements of the 
mathematics and science curriculum (Weiss, 1987; McKnight, 1987). Second, research has 
established links between tne use of certain instructional materials, such as textbooks and 
calculators, and student achievement at the secondary level (Dossey et al., 1988). Third, 
some instructional resources, such as computers and videodiscs, reflect educational 
innovations whose ultimate impact on the curriculum and achievement is unknown. It is 
important from a policy perspective to monitor the infusion of these reforms and their impact 
on instruction and outcomes. 

The major problem associated with developing indicators of instructional equipment 
and materials is the lack of a sound empirical basis for framing such indicators (Shavelson et 
al., 1987; Oakes iSid Carey, 1989). Research offers little guidance for deciding which 
resources to monitor or which features of the resources to assess to provide useful 
information. 

It is difficult to choose which instructional equipment and materials to monitor 
because we understand neither the specific role of instructional resoiirces in the overall 
curriculum nor their relationship to instruction and achievement. Certain resources 
represent enabling conditions that are necessary for instruction to occur one cannot teach 
laboratory science without laboratory facilities, nor (most would ai^e) can one teach 
computer programming without computers. These are clear candidates for inclusion in an 
indicator system. In addition, there is empirical evidence of links between other resources 
and student achievement at the secondary level, which would argue for the inclusion of these 
resources. For example, ^ands-on" science curricula (which rely upon laboratory activities) 
were found to be related to certain aspects of achievement (Shymansky, Kyle, and Alport, 
1983). Similarly, evidence from SIMS suggests that the lack of certain equipment and 
materials has been a source of problems to secondary school wachers (Crosswhite et al., 
1986). Supplemental research is needed to build a finar. basis for deciding which 
instructional resources, if any, to include in an indicator system. 
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Purthermore, it is not clear which aspects of the instructional resources should be 
monitored. There are at least four features that might be included in an indicator system: 
availability (Do schools provide particular equipment and materials?), access (Are teachers 
and students able to obtain the resources when needed?), use (What role do the resources 
play in learning and instruction?) and quality (Are the equipment and materials accurate 
and current from a mathematical and/or scientific perspective; are they well designed from a 
pedagogical standpoint, etc?). Each of these elements appears to provide meaningful 
information about the nature of the instructional resources that support the mathematics 
and science curriculum. 

Measuring the quality of instructional resources appears to be a particularly difficult 
task There have been a few attempts to gather information related to the quality of 
textbooks (Weiss, 1987; Crosswhite et al., 1986) and the condition of equipment (NELS), but 
this is uncommon. This lack of information is due. in part, to the difficulty of measuring 
quality. It may also be due to confusion about the meaning of the construct itself. 

Quality has both pedagogical and content dimensions. From a pedagogical standpoint, 
schools should use instructional resources that reflect the way students learn science and 
give meaning to mathematical and scientific concepts. Materials' design and development 
should be informed by the latest research on learning and cognition. From a content 
standpoint, instructional materials should be current and accurate. Scientific discoveries 
that affect our understanding of basic principles that are taught at the secondary level, that 
should be part of the framework of any scientifically literate citizen, or that represent new 
ways of investigating phenomena should be integrated into the curriculum as rapidly as 
possible. Students are not well served if the content of their science textbooks or materials 
are out of date or inaccurate. Thus, it would seem appropriate to include measures of quality 
as well as measures of availability, access, and use in an indicator system. 
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6. CONCLUSIONS 

Unfortunately, it is not possible to construct an adequate system of curriculum 
indicators based on existing data sources. The indicator patchwork pieced together from 
existing sources is incomplete or uneven in four important ways. First, the patchwork does 
not cover many important aspects of curriculum, i.e., there are significant gaps in our ability 
to describe opportunities to learn mathematics and science. For example, there are no data 
to describe the quality or the sequence of instruction in science or mathematics. 

Second, the patchwork is temporally uneven. Some data are current, others are 
almost a decade old; some are updated biennially, and some quadrennially and others are 
unlikely to be updated for a decade. For example, NAEP general information is collected 
every two years; NAEP mathematics and science assessments occur every four, six, or eight 
years. Longitudinal surveys, such as High School and Beyond (HSB) or NELS, are launched 
less than once a decade. The description of curriculum that can be created using data that 
are updated on a regular basis is quite rough. 

Third, the patchwork is of uneven quality. To fill in some gaps it is necessary to rely 
on data from less well*implemented or less rigorous surveys. For example, results based on 
SIMS or SISS are only suggestive of national trends because neither survey was based on a 
nationally representative sample of students. 

Fourth, measures drawn from different data sources are not alw.'^ys congruent, so it 
may not be possible to draw desired comparisons. For example, it is not possible to compare 
information on science laboratories with information on science courses because the data 
were not collected from the same schools. 

As a result, our knowledge of the status of mathematics and science curriculum in U.S. 
secondary schools is inadequate for effective policymaking. Reliable, valid data exist to 
describe some aspects of the curriculum (e.g., graduation requirements, course availability, 
and course completion), to track the status of some of these constructs over time, and to 
examine the distribution of some curriculum elements across schools and students. In 
contrast, little or no data are available to describe other important elements of the 
curriculum (e.g., the topics that are covered in mathematics and science classes and the 
manner in which they are presented), or the data that are available are unreliable, 
unrepresentative, or not collected on a regular basis. The greatest gaps in our knowledge of 
curriculum occur at the classroom level: little is known about the actual content of courses or 
the manner in which content is presented. At the present time, a patchwork indicator 
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system can provide a picture of curriculum that is adequate only if we are willing to ignore 
such classroom-level variation. 

The lack of information about coxu^se content is a serious deficiency. Measures of 
course completion do not adequately reflect students' exposure to specific mathematical and 
scientific knowledge and patterns of thought, nor do they reveal the full extent of differences 
in curriculum opportunities. This limits their value as a monitoring tool. Furthermore, 
ctirrent reform efforts focus on the content and process of mathematics and science 
education. Curriculum reform is designed to encourage students to reason logically, think 
scientifically, solve problems, communicate findings, etc. through the actual performance of 
scientific experiments and the application of mathematics to real problems. The curriculum 
patchwork that can be assembled ft^om existing data is insensitive to changes likely to be 
engendered by such reforms, so it is of limited value for monitoring the effects of these 
efforts. 

The partial picture that can be portrayed from existing data reveals that some 
students have access to broader curriculum opportunities than others, and these differences 
are not random variations but systematic differences associated with identifiable conditions. 
It is critical that such differences be more closely monitored through a mechanism such as an 
indicator system so that problems can be identified, addressed, and, hopefiilly, alleviated. 

RECOMMENDATIONS 

Although this analysis revealed serious deficiencies in our ability to describe 
mathematics and science curriculum and to monitor curriculum changes, most of these 
deficiencies can be remedied through additional research and data collection. There are four 
broad areas of action the National Science Foundation might consider to fill the gaps in the 
existing data network and to build a comprehensive curriculum indicator system. 

First, although existing sources provide basic data about graduation standards, course 
availability, and course completion, there are no assurances that these data will continue to 
be available on a regular basis in the future. It is likely that data on the first two of these 
features—graduation requirements and course availability—will continue to be collected (by 
NAEP, ECS, CCSSO, or other agencies), but this is not a certainty. Neither the National 
Assessment Governing Board nor the leadership of the Education Commission of the States 
has specific reasons to collect these data in a style and format appropriate for curriculum 
indicators. It might be prudent to act to ensure the continued availability of these two types 
of core data in an appropriate format. 
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Course completion measures pose a greater problem because complete data are not 
collected in any ongoing surveys. Only incomplete self-reported measures of course taking 
are available through NAEP. There are two ways this deficiency might be remedied— new 
transcript studies or modifications to NAEP. The most reliable course completion data come 
from transcript studies, and the most satisfactory solution would be to take actions to ensure 
that regular transcript studies were conducted. As an alternative, NAEP course-taicing 
measures could be expanded to provide a basis for course completion indicators. This would 
require extending the range of coxarses on which students were asked to report. It also would 
require supplemental research to validate these self-reported data against transcript-based 
results, because it would be unwise to rely on modified NAEP course-taking measures 
without such validation research. 

Second, the greatest gap in current curriculum data concerns course content. A 
comprehensive curriculimi indicator system should be able to describe the content of 
mathematics and science courses and how this content is presented to students. To 
accomplish this, measures of course content would have to be developed, validated, and 
incorporated into ongoing data-collection efforts. NSF already has fiinded promising 
research to investigate alternative coursework indicators at the eighth grade level, but much 
more research is needed to complete this developmental work and to broaden the scope to 
include subject matter content and instructional strategies at multiple grades. 

Third, an indicator system requires ongoing maintenance; it is not enough merely to 
collect data and compute indicators. Supplemental research is needed to validate and extend 
the information provided by ongoing data sources. Such research would include validation of 
specific indicator alternatives, examination of the relationship between curriculum measures 
and other student outcomes, and targeted studies of specific topics of interest within and 
across mathematics and science curriculum domains. The maintenance of an indicator 
system requires an ongoing commitment of resources for such supplemental development 
and validation research. 

Finally, more information is needed to describe the mathematics and science 
curriculum at the elementary and middle school levels. It is clear from SIMS and NELS that 
curriculum differentiation has begun already by the eighth grade; it would be valuable to 
understand much more about the presentation of mathematics and science prior to that 
grade level. Few sources exist to describe elementaiy mathematics and science curriculum, 
so much work would have to be done to fill this gap. It would be necessary to develop surveys 
to gather relevant data and to define and validate curriculum indicators based on these data. 
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The National Science Foundation has a number of options regarding curriculum 
indicators, from fully funding all four of the efforts described above to taking no actions at 
all. While all four components would be necessary to have an ongoing, comprehensive, and 
valid mathematics and science curriculum indicator system, they are not equally important. 
Moreover, the actions suggested above do not have equal priorities for NSF. 

It is likely that most of the desired information about graduation standards and course 
availability will continue to be collected by other organizations. These are the two areas of 
curriculum where the potential to build a valid indicator patchwork is the greatest. 
However, small changes and modifications to the work of ECS, NAEP, and GCSSO would 
increase the value of these data for use in an indicator system. NSF might try to influence or 
coordinate the design of these surveys so they better meet the needs of an indicator system. 

Producing appropriate course completion data may require a somewhat larger effort 
on the part of NSF. Althougji an expansion of NAEP course-taking measures conceivably 
could b** accomplished at little cost, it would have to be accompanied by validation research. 
Transcript studies are the preferred alternative, but such studies are expensive. However, 
since transcript studies provide data of value to many educational constituencies, it might be 
possible to develop a collaborative arrangement among educational agencies to share the 
costs of regular studies of this type. 

The most significant gap in our ability to describe mathematics and science curriculum 
involves course content NSF has already recognized the importance of this problem, and it 
has sponsored research to explore the development of coursework measures. These efforts 
satisfy a need that is largely unmet through other sources, and the agency should consider 
continuing or even expanding this work until the potential for such indicators is better 
understood. 

Another area in which NSFs efforts may jrield significant returns is the ongoing 
enhancement and validation of the basic curriculum indicators derived from other surveys. 
No other group is actively supporting this important work. Such research is necessary to 
maintain the quality of the existing incomplete patchwork, to validate additional secondary- 
level curricultmi indicators, and to provide information about other issues of interest within 
and across mathematics and science curriculum domains. 

Once a broad secondaiy-level curriculum indicator system is operational, attention 
should turn to curriculum opportunities in earlier grades. Existing data suggest that it is 
important to monitor students' exposure to mathematics and science at the elementaiy and 
middle school levels. However, by postponing development of elementary indicators for a 
time, future work on this topic can benefit from the research done at the secondary level. 
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Resources might be used more efficiently as a result. A study to investigate the costs and 
feasibility of developing elementary and middle school curriculum indicators in mathematics 
and science might be a reasonable first step toward a more complete set of elementary and 
middle school curriculum indicators. 
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Appendix A 

PRINCIPAL DATA SOURCES FOR BUILDING CURRICULUM INDICATOR PATCHWORK 

ADMINISTRATOR AND TEACHER SURVEY (ATS) FROM HIGH SCHOOL AND BEYOND 

ATS was designed to explore relationships suggested by research on effective schools 
using a broadly representative sample of teachers and students. The survey, which was 
funded by the Office of Research, U.S. Department of Education, and conducted in 1984 by a 
consortium of five federally supported research centers, was a supplement to the national 
longitudinal survey of high school students known as High School and Beyond. The 
nationally representative sample of teachers and administrators answered questions 
regarding goals, pedagogical practices, interpersonal relations, workloads, attitudes, etc. The 
data's greatest use in the present investigation related to graduation standards. To our 
knowledge, there are no plans to repeat ATS in the future. 

COUNCIL OP CHIEF STATE SCHOOL OFFICERS' STATE EDUCATIONAL INDICATORS 
(CCSSO) 

In 1984, CCSSO established the State Education Assessment Center to improve the 
collection and use of data on education by the states. The center has attempted to expand 
the breadth of data collected by states, improve the quality of the data that are collected, and 
facilitate the dissemination and use of these data. Toward this end CCSSO began publishing 
State Educational Indicators in 1987 as an annual compilation of data from states. Initially, 
the report included primarily demographic data, but in 1989 it was expanded to include 
information about school system accoimtability. Further expansion is planned for the future. 

EDUCATION COMMISSION OF THE STATES (ECS) SURVEY OF MINIMUM HIGH SCHOOL 
GRADUATION REQUIREMENTS 

Periodically, the Education Commission of the States siirveys state Departments of 

Education and compiles a summary of state minimum high school graduation requirements. 

ECS has been publishing the results of these sur ; eys as part of their series of Clearinghouse 

Notes since roughly 1983. ECS maintains an update file on each state and pxiblishes a new 

summary report when enough changes have been made to warrant it. The most recent 

summary was published in 1989. 
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HIQH SCHOOL TRANSCRIPT STUDY (HTS), 1987 

This study was conducted in 1987 for the U.S. Department of Education's National 
Center for Education Statistics (NOES). The purpose was to provide information for NCES 
and policy-makers regarding course offerings and course taking in the nation's secondary 
schools. An attempt was made to sample the same students who participated in the 1985- 
1986 National Assessment. While the exact sample could not be matched, students were 
sampled in the same manner from the same schools, so the results reflect the graduation 
status of students similar to those who participated in NAEP when they were in grade 11. 
The results were tabulated and reported in Westat, Inc. (1988). There are no plans to repeat 
this analysis on a regular basis. 

INTERNATIONAL ASSESSMENT OF MATHEMATICS AND SCIENCE (JAEP) 

lAEP involved representative samples of students from five countries and four 
Canadian provinces. The project was designed to capitalize on the content and experience of 
NAEP in the United States and used existing assessment questions and procedures to a large 
extent. The assessment focused on 13-year-old students from grades seven and eight. In 
addition to questions regarding mathematics and science, students were asked about their 
school experience and attitudes, and teachers rated students' exposure to the concepts tested 
by the items. Results were reported in Lapointe, Mead, and Phillips (1989). 

NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS (NAEF) 

NAEP is a biennial national survey of student performance in selected subjects 
founded by Congress and conducted under contract to the Department of Education. 
Originally designed to report only national and regional results at three age levels (9 year 
olds, 13 year olds and 17 year olds), NAEP has begun to report state-level results as well and 
has shifted its targeted grade levels to 4th grade, 8th grade, and 12th grade. In the past, 
many different subjects have been assessed, including reading, writing, social studies, 
history, science, mathematics, and art. Reading and writing have been assessed every cycle; 
math has been assessed approximately every other cycle. It appears that NAEP results will 
be available on an ongoing basis every other year, and the size and scope of the endeavor will 
increase. Achievement is the major focus of the NAEP assessments, and there has been 
considerable variation in the depth and consistency with which curriculum and instructional 
practice variables have been measured. Data reported in this study were drawn primarily 
from the 1985-1986 NAEP that included mathematics, science, and computer competence in 
addition to reading and writing. Students were sampled from grades 3, 7, and 11 (as well as 
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ages 9, 13, and 17). The results were reporU^d in Dossey et al (1988); Martinez and Mead 
(1988); and MulUs and Jenkins (1988) among others* 

NATIONAL EDUCATION LONGITUDINAL STUDY (NELS) 

NELS is the latest in the longitudinal education surveys sponsored by the U.S. 
Department of Education's National Center for Education Statistics. (Previous longitudinal 
surveys included the National Longitudinal Study and High School and Beyond.) NELS 
began with an eighth grade cohort of students in 1988 and will continue to follow that group 
of students with biennial surveys through high school and into adult life. Information 
gathered fi^om students regarding their knowledge and educational e:q;)eriences will be 
supplemented with data from parents, teachers, and school administrators. Among the 
reports describing conditions in grade eight is Horn and Hafher (in press). 

NATIONAL SURVEY OF SCIENCE AND MATHEMATICS EDUCATION (NSSME) 

NSSME was a national survey of the status of mathematics and science education 
conducted in 1985-1986 under the auspices of the National Science Fouidation. The survey 
focused on mathematics and science curriculum, teachers, and resources; NSSME did not 
assess student achievement. The study was designed to update the results of the earlier 
1977 National Survey of Science, Mathematics, and Social Studies Education and detect 
trends in science and mathematics education. The results of NSSME were reported in Weiss 
(1987). There are no specific plans to repeat the survey on a regular basis. 

SCHOOL AND STAFFING SURVEY (SASS) 

The Center for Education Statistics (CES) of the U.S. Department of Education 
initiated the School and Staffing Survey in 1387-1988 to measure critical aspects of teacher 
supply and demand, the composition of the administrator and teacher workforce, and the 
status of teaching and schooling. The survey represents a revision of earlier surveys that had 
been conducted separately for public and private school staff. While the main focus is on 
teacher supply and demand, there are questions regarding instructional practices, course 
offerings, etc. that are relevant to curriculum*indicator development. SASS is designed to be 
an ongoing national data collection effort. 

SECOND INTERNATIONAL MATHEMATICS STUDY (SIMS) 

SIMS was conducted imder the auspices of the International Association for the 
Evaluation of Educational Achievement (lEA) in 1981-1982. llie purpose of the study was to 
investigate ways u\ which mathematics was tau|^t, to describe student attitudes and 
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achievement, and to relate outcome variables to curriculum and teaching practices. The 
study represented an international collaboration among two dozen countries. In the United 
States, the project was funded by the National Institute of Education, the National Science 
Foundation, and the National Center for Education Statistics; it was coordinated by a 
National Mathematics Committee consisting of mathematicians and scholars from major 
universities. The U.S. National Coordinating Center was located at the Univer'^ity of Illinois. 
Two populations were sampled: students in grade 8 and students taking precalculus/calculus 
(typically in grade 12). The study provides a rich and complex array of data regarding 
mathematics curriculum instruction and achievement. Its major drawback is that the 
sample of students in the United States who participated in the study was small and 
unrepresentative. Therefore, the results were not generalizable, though they were quite 
suggestive of the condition of mathematics education in the United States. Results have 
been published in Travers and Westbury (1989); Crosswhite et al. (1986); McKnight et al. 
(1987); Burstein (1991) and other sources. The first international survey was conducted in 
1964, and plans are under way for a third study to be conducted soon. 

SECOND lEA SCIENCE STUDY (SISS) 

Like its counterpart SIMS, SISS was an international comparative study of science 
achievement and instruction conducted under the auspices of the International Association 
for the Evaluation of Educational Achievement (lEA) in 1981-1982. Results have been 
published in Jacobson et al. (1987), Jacobson and Dcran (1985), and other sources. 

SCHOOL REFORM ASSESSMENT PROJECT (SRA) 

SRA was a two-year exploratory design project conducted by RAND and the UCLA 
Center for the Study for Evaluation and funded by the Office of Educational Research and 
Improvement, U.S. Department of Education, The purpose of the study was to refine the 
technical quality of existing coursework indicators and to design indicators that would meet 
the information needs of pohcymakers to measure the effects of curriculum policies. The 
study undertook several benchmarking procedures, including interviews, transcript analyses, 
and evaluations of course materials in addition to reviewing data from existing sources. The 
results were reported in McDonnell et al. (1990). 
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Appendix B 

NAEP VARIABLES WITH HIGH RATES OF MISSING V/^LUES IN 1985*1986 



The following is a partial list of variables for which missing values were reported in 10 
to 15 percent of the schools in the 1985-1986 NAEP sample: 

• Availability of the following courses: algebra II, advanced geometry, AP biology, 
AP chemistry, physics I, and programming I; 

• Percent of students enrolled in remedial mathematics courses and in remedial 
reading courses; 

• Percent of students classified as ESL; 

• Percent of students who drop out; 

• The number of semesters of the following courses required for graduation: 
English, history, mathematics, and science; 

• Use of ability grouping in English, in history, in mathematics, and in Spanish; 

• Presence of specialized science laboratories. 

The following is a partial list of variables for which missing values were reported in 16 
to 20 percent of the schools in the 1985-1986 NAEP sample: 

• Percent of students recei^nng subsidized lunch; 

• Availability of the following courses: physics II, statistics, and programming II; 

• Percent of students in academic, general, and vocational programs; 

• Presence of classroom laboratories. 
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AppendlxC 
ADDITIONAL TABLES AND FIGURES 



Table C*l 

State-Imposed Regular Graduation Requirements in Mathematics 

Percentage of States* 



xears 
Required 


1974 


1980 


1983 


1985 


1987 


1989 


1990 


0.0 


29 


27 


24 


12 


12 


10 


12 


0.5 
















1.0 


51 


53 


18 


4 


4 


2 


2 


1.5 


4 


2 












2.0 


14 


16 


49 


65 


61 


59 


63 


2.5** 






2 


2 


4 


6 


2 


3.0 


2 


2 


8 


18 


20 


22 


22 


3.5 
















4.0 

















*50 states plus the District of Columbia. 

**Soine states required an additional year of course work in either math or science. 
For the purpose of this table such a requirement was counted as one-half year in each 
subject. 

SOURCES: RAND Ubulations of data from the National Association of Secondary 
School Principals, 1975 & 1980; Education Commission of the States, 1983 & 1990; 
Center for Education Statistics, 1987; National Center for Education Statistics, 1989; 
Clune, 1989. 



Table 

State-Imposed Regular Graduation Requirements in Science 



Percentage of States* 



Required 


1974 


1980 


1983 


1985 


1987 


1989 


1990 


0.0 


28 


28 


24 


12 


12 


10 


12 


0.5 
















1.0 


53 


53 


29 


16 


10 


10 


8 


1.5 


4 


2 












2.0 


14 


18 


39 


63 


67 


67 


71 


2.5** 






4 


2 


4 


6 


2 


3.0 






4 


8 


6 


6 


8 


3.5 
















4.0 

















*50 states plus the District of Columbia. 

**Some states required an additional year of coursework in either math or science. 
For the purpose of this table such a requirement was counted as one-half year in each 
subject. 

SOURCES: RAjND tabulations of data from the National Association of Secondary 
School Principals, 1976 & 1980; Education Commission of the States, 1983 & 1990; 
Center for Education Statistics, 1987; National Center for Education Statistics, 1989; 
Clune, 1989. 
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Fig. C«l-- Science Course Availability by Grade Level Enrollment 
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SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 

Fig« C*2 — Computer Science Course Availability by Grade Level Enrollment 
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SOURCE: RAND tabuiati^xis of the 1985-1986 National Assessment of Educational Progress 

Fig« C«3 — Science Course Availability by Size and Type of Commimity 
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SOURCE: RAND tabulations of the 1985-1986 Nationai Assessment of Educational Progress 



Fig. 0.4 — Computer Science Course Availability by Size and Type of Community 



i 
I 



ERIC 



-95- 



UK 0% - 24% Ql 25% - 49% 1 1 50% ^ 74% | 1 75% - 1 00% 
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Course title 

SOURCE: RAND tabulations of the 1985-1966 National Assessment of Educational Progress 
Fig. Mathematics Course Availability by Percent Non-Asian Minority Enrollment 
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RAND tabulations of the 1985-1986 National Assessment of Educational Progress 
C*6 — Science Course Availability by Percent Non-Asian Minority Enrollment 



ERIC 



-97- 




Computer literacy Programming I Programming II 



Course title 

SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 
Fig. €.7 — Computer Science Course Availability by Percent Non-Asian Minority Enrollment 
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SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 

Fig, C.8— 'Mathematics Course Availability by Percent Receiving Subsidized Lunch 
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Course title 

SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 

Fig. Science Course Availability by Percent Receiving Subsidized Lunch 
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SOURCE; RAND tabulations of the 1985-1986 National Assessment of Educational Progress 
Fig. Computer Science Course Availability by Percent Receiving Subsidized Lunch 
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Algebra II Trigonometry Calc/precalc 

Course title 

SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 



Fig. C.11— Mathematics Course Availability by School Type 
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SOURCE: RAND tabulations of the 19d&-1986 National Assessment of Educational Progress 
Fig. C»12 — Science Course Availability by School Type 
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SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 

Fig. C.13— Computer Science Course Availability by School Type 
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Fig. C.14 — College-Equivalent Course Availability by Size and Type of Community 




^4 



-105- 



im 0% - 24% 


25% - 49% 1 1 50% - 74% | 1 75% - 1 00% 





o 
o 
sz 
o 

CO 



c 

0) 

o 
a. 




Mathematics 



Biology Chemistry 
AP or second-year course in subject 
SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 



Physics 



Fig. C.15 — CoUege-Equivalent Course Availability by Percent Non-Asian 
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Fig* C«16 — College*Equivalent Course Availability by Grade Level Enrollment 
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Fig. C.17 — College-Equivaient Course Availability by School Type 



1?' 



- 108 



[1-30 
I Students 



I 31 - 90 
I students 



91 - 240 
Students 



□ 



241 + 
students 



100 



■D 

3 

GO 
(D 

2 



c 
o 

Q- 




Geometry 



Algebra I 
Course title 



Trigonometry 



SOURCE: RAND tabulations of the 1985-1986 National Assessment of Educational Progress 
Fig. C.1&— Mathematics Courses Taken Through Grade 11 by Grade Level Enrollment 
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Fig. C.20 — Science Courses Taken Through Grade 11 by Percent Receiving 

Subsidized Lunch 
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